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provides  the  potential  pitfalls  that  an  AC  AT  ID  program  can  face  as  it  progresses  from 
Technical  Development  and  into  System  Design  and  Demonstration  (SDD).  The  inquiry  will  be 
accomplished  by  examining  the  Aerial  Common  Sensor  (ACS)  (ACAT  ID)  Airborne 
Intelligence  Surveillance  and  Reconnaissance  program  as  a  representative  case  study. 

The  Department  of  Defense’s  Acquisition  Process  has  been  called  into  question  several 
times  over  the  last  twenty  years.  The  induced  acquisition  reform  cycles  and  changes  to  the 
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the  process  and  checks  are  designed  to  ensure  program  success  measured  by  adherence  to 
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breach  one  or  more  of  these  measures.  In  the  first  quarter,  FY06,  of  the  eighty-five  programs 
reported  to  Congress  in  the  Selected  Acquisition  Report,  47%  reported  Nunn-McCurdy  unit  cost 
breaches. 

This  analysis  reviews  the  DoD  acquisition  policy  and  reform  initiatives  and  then 
researches  the  current  DoD  acquisition  policy  and  process  as  it  was  applied  to  the  ACS  program. 
This  paper  then  researches  the  current  process  pitfalls  that  affected  the  ACS  Program.  This  is 
followed  by  an  analysis  of  the  recent  Defense  Acquisition  Process  Assessment  and  discusses 
whether  it  will  be  effective  in  causing  meaningful  reform.  This  paper  then  suggests  changes  that 


should  be  made  to  the  acquisition  process  and  provides  insights  that  can  be  used  to  help  future 
programs  avoid  the  pitfalls  that  the  ACS  program  faced  during  its  contract  execution. 
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PITFALLS  OF  THE  DEFENSE  ACQUISITION  PROCESS  -  EXPERIENCE  IS 

THE  KEY  TO  SUCCESS 

Introduction 

“There  is  a  growing  and  deep  concern  within  the  Congress  and  within  the  Department  of 
Defense  (DoD)  Leadership  Team  about  the  DoD  acquisition  process.  Many  programs 
continue  to  increase  in  cost  and  schedule  even  after  multiple  studies  and 
recommendations  that  span  the  past  15  years.  In  addition,  the  DoD  Inspector  General  has 
recently  raised  various  acquisition  management  shortcomings.” 

—  Gordon  England,  Deputy  Secretary  of  Defense,  June  2005. 

The  effectiveness  of  the  Department  of  Defense’s  (DoD)  acquisition  process  has  been 
called  into  question  many  times  over  the  last  35  years  due  to  the  inability  of  the  process  to 
consistently  deliver  required  capability  to  the  warfighter  on  cost  and  within  schedule  as 
estimated  and  described  in  a  program’s  start-up  documentation  (Acquisition  Program  Baseline  - 
APB)[11.  Due  this  inability  to  execute  a  program  on  targeted  cost  and  within  established 
schedule,  Congress,  the  warfighter  and  the  American  people  have  lost  confidence  in  the  DoD’s 
ability  to  effectively  manage  its  investment  account  budget.  This  in-turn  causes  calls  for  more 
and  varied  oversight  and  regulation  of  the  acquisition  process  for  large  Acquisition  Category  ID 
(AC AT  ID)  programs  resulting  in  additional  burdens  on  a  programs  ability  to  effectively  execute 
system  development  and  adding  to  the  potential  of  a  program  not  achieving  its  APB  [2],  From  a 
historical  perspective,  this  appears  to  be  a  cycle  that  DoD  is  unable  to  break  but  one  that  now 
more  than  ever  must  be  broken  based  on  ongoing  competing  needs  for  the  finite  budget 
available  (3  k 

This  continual  problem  with  the  DoD  acquisition  process  has  induced  several  acquisition 
reform  cycles  and  ultimately  changes  to  the  guiding  acquisition  directives  over  the  years 
resulting  in  updates  to  the  material  acquisition  process  for  ACAT  ID  programs[4].  These  process 
changes  have  for  the  most  part  centered  on  attempting  to  provide  confirmation  to  the  Defense 
Department  and  Services  of  a  program’s  preparedness  to  proceed  to  a  program  start  (Milestone 
B)  decision  and  into  the  development  phase  -  System  Design  and  Demonstration  (SDD). 
Specifically,  the  recent  changes  direct  the  use  of  best  practices[51  that  are  captured  in  the 
approximately  thirty-five  separate  documents  that  the  program  manager,  as  well  as  the  service 


and  OSD  staffs  must  produce  and  provide  to  the  program’s  reviewing  officials  as  it  winds  it  way 
through  the  briefing  and  approval  process  to  a  Defense  Acquisition  Board  (DAB)  Milestone  B 
decision[6].  While  the  adoption  and  use  of  best  practices  as  described  in  the  current  DoD 
Directive  5000.1  and  Instruction  5000.2  are  designed  to  ensure  program  success  as  measured  by 
delivering  the  required  capability  to  the  user  at  the  cost,  and  within  the  schedule  outlined  in  the 
APB,  programs  continue  to  significantly  breach  one  or  more  of  these  measures.  In  the  first 
quarter,  FY06,  of  the  eighty-five  ACAT  ID  programs  reported  to  Congress  in  the  Selected 
Acquisition  Report,  forty  programs  reported  Nunn-McCurdy  unit  cost  breaches  with  twenty-five 
of  these  reporting  greater  than  50%  unit  cost  growths[71. 

The  best  practices  required  by  the  DoDI  5000.2  documentation  are  based  on  prior 
acquisition  reform  studies  that  called  for  needed  change  in  the  process  based  on  lessons  learned 
through  the  analysis  of  prior  programs  that  were  unable  to  deliver  a  capability  on  cost  or  within 
schedule  as  estimated  at  program  start.  While  these  required  documents  cause  a  program 
manager  and  Service/OSD  staffs  to  think  about  a  particular  best  practice  process  area,  what  they 
appear  to  lack  is  a  treatment  of  the  lessons  learned  and  the  pitfalls  faced  by  the  previous  program 
that  the  best  practice  is  designed  to  mitigate.  Therefore,  while  a  program  manager  and  the  staffs 
of  Service  and  OSD  reviewing  officials  must  create  documents  because  of  the  directive’s  call  for 
adherence  to  best  practice  processes,  whether  the  PM  or  Service/OSD  staffs  fully  recognize  the 
pitfalls  that  the  best  practice  was  designed  to  ameliorate  is  not  assured. 

The  Aerial  Common  Sensor  (ACS)  program  is  an  ACAT  ID  program  due  to  its  expected 
cost  and  joint  interest.  The  intent  of  this  program  is  to  fill  a  capabilities  gap  that  will  soon  exist 
for  both  the  Army  and  Navy  as  the  current  Airborne  Intelligence  Surveillance  and 
Reconnaissance  aircraft  of  each  service  reach  the  end  of  their  useful  lives[8].  ACS  had 
successfully  accomplished  the  Component  Advanced  Development  (now  called  Technology 
Development  (TD))  phase  of  its  life-cycle  a  month  after  the  publishing  of  the  current  DoD  5000 
series  in  May  of  2003.  Hence,  the  best  practice  processes  and  documentation  requirements  that 
the  program  followed  as  it  proceeded  from  TD  to  a  DAB  Milestone  B  decision  are  those  outlined 
in  the  current  DoDI  5000.2.  However,  after  a  successful  DAB  decision  to  proceed  into  System 
Design  and  Demonstration  (SDD)  and  within  four  months  after  contract  award,  the  program 
began  suffering  from  system  design  issues  that  ultimately  would  impact  the  program’s  ability  to 
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meet  its  documented  system  delivery  date.  By  September  2005,  the  extent  of  the  design  issues 
and  estimated  schedule  slip  had  grown  to  the  point  that  the  program  was  called  before  a  special 
Army  Systems  Acquisition  Review  Council.  This  council  recommended  contract  termination 
that  occurred  on  12  January  2006  only  a  year  and  a  half  after  the  SDD  program  started[9].  While 
there  is  disagreement  on  whether  the  ACS  program  fully  followed  all  the  best  practice  processes 
as  outlined  in  the  DoD  5000  instruction,  what  becomes  apparent  upon  analysis  of  the  programs 
history  is  that  even  though  the  best  practice  documentation  was  fully  developed,  the 
understanding  of  the  extent  of  the  potential  pitfalls  of  this  phase  of  the  program’s  life-cycle  were 
not  readily  recognized. 

Capturing  and  understanding  best  practices  appears  to  have  been  gaining  attention  across 
the  Executive  Branch  of  Government  over  the  last  four  years.  As  DoD  and  the  rest  of  the 
Executive  Branch  work  to  transform  themselves  into  organizations  that  are  more  efficient,  the 
need  to  change  the  way  they  do  business  has  centered  on  whether  the  current  processes  are  based 
on  best  practices  resulting  from  lessons  learned  from  the  pitfalls  faced  by  previous  efforts.  For 
instance,  The  SDD  phase  of  a  program’s  life-cycle  received  much  attention  in  the  February  2006 
Defense  Acquisition  Performance  Assessment  (DAPA)  study  accomplished  at  the  request  of 
Acting  Deputy  Secretary  of  Defense  Honorable  Gordon  England.  In  this  study,  acquisition 
process  reform  is  discussed  from  a  holistic  “big- A”  perspective  including  stake-holders  such  as 
the  requirements  community  and  Congress  not  traditionally  considered  part  of  the  “little-a”  DoD 
acquisition  process.  This  perspective  of  reform  discussion  brings  to  light  pitfalls  acquisition 
programs  face  in  the  SDD  phase  of  a  program’s  life-cycle.  Many  of  these  pitfalls  are  the  same  as 
those  that  were  experienced  by  the  ACS  program.  The  study  then  makes  recommendations  that 
focus  on  accounting  for  these  pitfalls  with  the  ultimate  goal  being  speeding-up  the  delivery  of 
capabilities  to  the  warfighter [10], 

This  reform  study  is  part  of  a  larger  on-going  business  transformation  effort  which  was 
outlined  in  2002  by  President  Bush  in  his  President’s  Management  Agenda.  This  agenda  calls  for 
all  federal  agencies  to  be  customer  focused  and  to  establish  a  set  of  metrics  by  which  to  measure 
the  success  of  their  best  practice  processes  designed  to  reduce  time  for  delivery  of  capability  or 
products  to  the  customer.  The  focus  again  is  understanding  and  avoiding  pitfalls  that  would 
negatively  impact  the  ability  to  accurately  estimate  a  program’s  cost  and  schedule  to  deliver 
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required  capability  to  the  user.  This  business  transformation  effort  also  requires  that  the 
processes  be  reviewed  and  updated  on  a  regular  basis  in  an  effort  to  continually  decrease  the  time 
needed  to  deliver  product  to  the  user  while  also  reducing  cost[  11]. 

Considering  the  apparent  focus  of  the  Executive  Branch  and  DoD  on  best  practices,  why 
is  it  that  so  many  ACAT  ID  programs  continue  to  be  unable  to  accomplish  their  stated  goals 
with  respect  to  cost,  schedule  and  performance?  From  the  analysis,  it  appears  that  the  current 
DoD  acquisition  process  and  support  structure  for  ACAT  ID  programs  while  being  committed  to 
the  use  of  best  practice  processes  still  have  not  incorporated  a  unified  lessons  learned 
methodology  for  capturing,  disseminating,  and  mitigating — through  continued  best  practice 
process  improvement — potential  pitfalls  that  an  ACAT  ID  program  entering  SDD  could  face. 
“The  current  system  is  focused  on  programs,  not  on  improving  and  standardizing  the  processes 
of  acquisition;  it  inhibits  rather  than  promotes  steady  improvement  in  achieving  program 
success” [1 2] . 

This  paper  researches  the  question  above  and  provides  suggestions  for  the  types  of 
pitfalls  that  a  program  can  encounter  which  could  impede  the  path  to  successful  delivery  of 
warfighter  capability  on  target  cost  and  within  schedule.  To  accomplish  this,  the  ACS  program 
will  be  used  as  a  case  study  of  a  representative  ACAT  ID  SDD  program  that  even  though  it 
followed  the  current  DoD  acquisition  process  fell  victim  to  the  pitfalls  that  could  have  been 
mitigated  had  they  been  known  before  hand.  Additionally,  the  paper  will  make  recommendations 
on  how  to  improve  the  current  acquisition  process  by  developing  a  unified  lessons  learned 
methodology. 

Acquisition  Reform  and  DoDD/1 5000.1&.2 

“The  unpredictable  nature  of  Defense  programs  can  be  traced  to  instabilities  in  the 
broader  acquisition  system.  Fundamentally  reshaping  that  system  should  make  the  state 
of  the  Department’s  major  acquisition  programs  more  predictable  and  result  in  better 
stewardship  of  the  U.S.  tax  dollar.  There  are  several  ongoing  reviews  of  defense 
acquisition  improvements  being  conducted  both  within  and  outside  the  Department  in  an 
effort  to  address  these  issues.  Their  results  will  inform  the  Department’s  efforts  to 
reshape  defense  acquisition  into  a  truly  21st  Century  process  that  is  responsive  to  the 
joint  warfighter.” 

—  Quadrennial  Defense  Review  Report  2006 
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General  Accountability  Office  (GAO)  studies  have  shown  that  acquisition  best  practices 
in  commercial  industry  are  knowledge  based.  Decisions  concerning  the  use  of  technology, 
expected  capability,  program  cost,  program  structure  and  moving  from  one  phase  of  the 
acquisition  process  to  the  next  are  based  on  information  gained  thru  the  use  of  metrics  that 
indicate  to  a  PM  and  corporate  leadership  when  the  unknown  risks  have  been  reduced  to  an 
acceptable  level  to  proceed.  This  translates  into  confidence  in  a  products  expected  development 
cost  and  schedule  which  ultimately  reduces  overall  cycle  time  to  bring  a  product  to  market.  This 
knowledge-based  approach  to  acquisition  has  become  a  process  that  is  depicted  graphically  in 
Figure  1. 
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Figure  1.  Knowledge  Based  Acquisition  Process[13]. 


In  this  knowledge-based  acquisition  process,  a  program  only  moves  forward  to  the  next 
phase  when  it  has  met  certain  controls/addressed  certain  potential  pitfalls  that  would  otherwise 
leave  unaccounted  for  risk  in  the  program  that  could  result  in  unexpected  cost  and  schedule 
growth.  Specifically,  at  Knowledge  Point  1  (program  start)  “requirements  and  technology  are 
matched”)  14].  Hence,  a  program  must  demonstrate  the  maturity  of  the  technology  to  be  used 
with  which  to  develop  the  product.  If  the  technology  has  not  reached  a  maturity  level  at  which 
the  risk  to  proceed  to  product  development  is  considered  low,  then  either  the  technology  is  first 
matured  before  proceeding  or  the  requirements  are  phased  such  that  the  current  technology 
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delivers  a  portion  of  the  products  expected  capability.  Technology  maturity  is  paramount  at 
Knowledge  Point  1 .  The  expected  level  of  maturity  before  a  technology  is  considered  ready  to  be 
incorporated  into  a  product  design  is  one  that  has  moved  “from  a  concept  to  a  feasible  invention 
to  a  component  that  must  fit  onto  a  product  and  function  as  expected”[151.  Using  the  technology 
readiness  scale  introduced  by  the  National  Aeronautics  and  Space  Administration  and  now 
adopted  by  DoD,  this  level  of  maturity  would  be  at  a  level  eight.  Technology  Readiness  Level 
eight  (TRL  -  8)  “is  a  technology  that  has  been  proven  to  work  in  its  final  form  and  in  its  intended 
operating  conditions.  A  radio  at  this  level  would  have  been  installed  in  the  instrument  panel  in 
the  aircraft  cockpit,  integrated  with  other  aircraft  systems,  and  flown  under  all  expected 
conditions”  [161. 

Beyond  the  control  of  technology  maturity,  there  are  several  other  key  controls  in  place 
in  the  “Best  Practices  (acquisition)  Model”  used  by  commercial  firms  successful  in  product 
development.  These  controls  all  further  the  level  of  knowledge  of  a  products  ability  to  meet 
requirements  at  a  particular  cost  and  schedule  by  forcing  potential  pitfall  areas  to  be  eliminated 
before  commitment  to  product  development.  Specifically  these  are:  “Ensuring  that  requirements 
for  the  product  are  informed  by  the  systems  engineering  process;  Establishing  cost  and  schedule 
estimates  for  product  based  on  knowledge  from  preliminary  design  using  systems  engineering 
tools;  Conducting  decision  reviews  for  program  launch;  and  that  the  producer  has  completed  a 
preliminary  design  of  the  product”[17]. 

Knowledge  Point  2  occurs  when  the  integration  of  the  technologies  is  complete  and 
before  the  program  moves  into  product  demonstration.  As  with  the  previous  knowledge  point, 
there  are  several  controls  in  place  to  ensure  that  the  product  design  is  at  a  level  of  maturity  to 
instill  confidence  in  the  corporate  review  board  that  product  development  is  ready  to  proceed. 

The  key  metric  at  this  point  as  stated  in  Figure  1  is  “percent  of  (product  design)  drawings 
complete”.  Specifically,  the  expected  percentage  of  drawing  complete  at  this  point  is  90%.  Based 
on  this,  the  design  is  expected  to  be  stable  and  demonstrated  through  prototype  testing  that  it 
meets  requirements[18]. 

Knowledge  Point  3  occurs  when  the  product  is  ready  to  begin  production.  At  this  point, 
production  controls  are  in  place  that  will  ensure  that  the  product  will  be  “manufactured  within 
cost,  schedule  and  quality  targets.”  Also,  by  this  time  the  product’s  reliability  is  known  through 
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demonstration.  The  key  metric  for  this  knowledge  point  is  the  “percent  of  key  production 
processes  in  control.”  Specifically,  leading  manufacturers  expect  to  have  all  of  their  key 
production  processes  under  statistical  process  control  prior  to  entering  into  production  “such  that 
the  quality,  volume,  and  cost  of  their  output  (have)  proven  acceptable”[19]. 

While  using  knowledge  based  acquisition  practices  was  a  focus  of  the  DoD  5000  series 
documentation  published  in  May  2003,  the  idea  of  using  knowledge  based  best  practices  in  DoD 
system  acquisition  is  not  a  new  concept.  Several  of  the  controls  discussed  above  were 
recommended  over  20  years  ago  in  the  June  1986  Packard  Commission’s  "Presidents  Blue 
Ribbon  Commission  on  Defense  Management”.  Here  the  Commission  looked  at  successful 
commercial  manufacturers  and  government  programs  and  distilled  best  management  practices 
that  led  to  products  being  brought  to  the  user  on  cost,  and  within  schedule.  With  respect  to 
technology  maturity  the  commission  recommended  developing  “subsystems  and  components 
independent  of  the  development  of  a  weapon  system”  and  the  use  of  .  .prototypes  and  less 
reliance  on  paper  studies.”  The  Commission  also  recommended  that  a  new  acquisition  policy 
should  assure  that  “maintainability,  reliability,  etc”  are  provided  by  “other  means  than  detailed 
documentation  by  contractors  as  part  of  design  proposals.”  The  study  recognized  that  “full-scale 
development  of  a  new  weapon  system  is  the  single  most  critical  step  in  the  acquisition  process. 

At  this  point,  a  number  of  fundamental  decisions  must  be  made:  whether  to  undertake  a  new 
development  or  adapt  an  existing  system,  how  far  to  push  the  new  technology  being  incorporated 
in  the  system,  what  cost  and  schedule  to  authorize,  and  what  the  management  structure  will  be. 
Misjudgment  about  any  of  these  items  can  start  a  program  off  on  a  course  that  dooms  it  to 
failure”[201.  Therefore,  knowledge  is  needed  to  ensure  that  the  right  decisions  are  made  so  the 
program  starts  on  a  path  to  success.  However,  in  the  July  1986  GAO  report  discussing  Defense 
Acquisition  Improvement,  the  study  team  noted  that  DoD  was  not  executing  its  materiel 
acquisition  process  based  on  best  practices  as  recommended  by  the  Packard  Commission. 
Specifically,  they  stated  that  DoD’s  "inability  to  submit  realistic  and  affordable  defense 
programs  and  budgets  to  the  Congress  for  the  development  and  procurement  of  weapon  systems" 
has  been  "because  they  have  not  always  included  all  expected  costs,  or  provisions  for  the 
technological  risks  associated  with  acquisition  of  high  technology  weapons”[21]. 
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Seven  years  later,  the  1993  the  Defense  Science  Board  recommended  as  part  of  its  overall 
study  entitled  Defense  Acquisition  Reform  that  DoD  should  increase  its  use  of  commercial  best 
practices  to  streamline  the  acquisition  process  in  an  effort  to  reduce  defense  acquisition  program 
cost  and  schedule.  Specifically,  the  study  encouraged  the  increased  use  of  “commercial  and 
commercial-like”  [best]  practices  to  the  point  that  “no  systematically  applied  unique  accounting 
practices,  specifications,  procurement  requirements,  reporting  systems,  and  management 
practices  would  be  required  beyond  those  normally  practiced  in  US  industry.”  Here  the  focus 
was  on  allowing  industry  to  more  openly  participate  in  the  defense  acquisition  process  and  to 
open  DoD  to  the  use  of  commercial  based  products.  The  idea  being  that  by  doing  this  the  success 
achieved  by  the  commercial  industry  acquisition  process  would  translate  to  the  defense 
acquisition  process  [22]. 

In  June  1994  the  Under  Secretary  of  Defense  for  Acquisition  Logistics  and  Technology 
(Acting)  and  the  Assistant  Secretary  of  Defense  for  Command  Control,  Communications  and 
Intelligence  co-signed  a  memorandum  to  the  Services  entitled  Software  Acquisition  Best 
Practices  Initiative.  This  memo  directed  the  identification  of  “criteria-based  practices”  used  by 
successful  software  development  efforts  in  both  the  Government  and  civilian  sector  that  could  be 
disseminated  and  used  by  all  software  development  efforts  in  DoD.  The  goal  of  this  initiative 
was  to: 

“  -  Focus  the  Defense  acquisition  community  on  employing  effective,  high-leverage 
software  acquisition  management  practices; 

-  Enable  Program  Managers  to  focus  their  software  management  efforts  on  producing 
quality  software,  rather  than  on  activities  directed  towards  satisfying  regulations  that  have 
grown  excessively  complex  over  time; 

-  Enable  Program  Managers  to  exercise  flexibility  in  implementing  best  practices  within 
disparate  corporate  and  program  cultures;  and, 

-  Provide  Program  Managers  and  staff  with  the  training  and  tools  necessary  to  effectively 
use  and  achieve  the  benefits  of  these  practices”[23]. 

Since  1996  GAO  has  recommended  the  use  of  corporate  best  practices  within  DoD  to 
improve  the  acquisition  process.  Specifically,  the  GAO  recommended  the  use  of  knowledge 
based  best  practices  and  metrics  (controls  which  eliminate  pitfalls/risk)  to  be  established  that 
measure  the  attainment  of  knowledge  levels  before  proceeding  with  the  commitment  of  product 
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development.  The  rationale  being  that  the  use  of  these  best  practice  acquisition  processes  have 
been  proven  in  industry  to  be  highly  effective  in  delivering  products  to  market  on  cost  and  within 
schedule,  and  by  implementing  these  practices,  DoD  could  also  benefit[24][251. 

In  the  2001  DoD  Quadrennial  Review,  the  theme  was  transformation  of  the  DoD  business 
practices  to  meet  future  challenges.  Action  items  were  to  realign  services  to  a  joint  focus,  reduce 
institutional  cost  and  reduce  cycle  time  delivery  of  capability  to  the  warfighter.  To  accomplish 
this,  one  of  the  needed  improvement  areas  called  for  in  the  review  was  reform  of  the  DoD 
acquisition  process.  By  reforming  the  DoD  acquisition  process  along  with  other  areas  the  review 
stated  “...truly  dramatic  improvements  in  future  joint  operational  effectiveness  (could)  be 
achieved”  [261. 

Current  Defense  Acquisition  System  -  DoD  5000. 1/.2  and  Guidebook 

It  is  the  history  described  above  of  calls  for  acquisition  reform  that  set  the  stage  for 
current  DoD  Acquisition  Process.  The  current  process  was  established  in  May  2003  with  the 
publishing  of  the  updated  Department  of  Defense  Directive  (DODD)  5000. 1  and  Department  of 
Defense  Instruction  (DODI)  5000.2.  A  quick  review  of  the  current  Defense  Acquisition  System, 
shows  that  DoD  heeded  the  recommendations  for  reform  and  established  an  acquisition  process 
focused  on  the  commercial  best  practice  of  knowledge -based  acquisition.  Like  the  commercial 
Best  Practices  Model,  the  current  DoD  process  calls  for  a  phased  approach  to  weapon  system 
acquisition  that  requires  the  attainment  of  knowledge  about  the  system  to  be  built  before 
committing  to  product  development  or  product  production.  This  concept  consists  of  five  phases: 
Concept  Refinement,  Technology  Development,  System  Development  and  Demonstration, 
Production  and  Deployment,  and  Operations  and  Support.  The  process  is  graphically  portrayed 
in  Figure  2(27] . 
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Figure  2.  Defense  Acquisition  Management  Framework. 

Before  each  phase  is  entered  there  is  a  decision  point  at  which  the  Milestone  Decision 
Authority  (MDA)  must  give  approval  before  the  program  is  allowed  to  move  into  the  next  phase 
of  the  acquisition  process.  These  decision  points  are  Milestone  (MS)  A,  B  and  C  and  mark  the 
transition  from  the  Concept  Refinement  to  the  Technology  Development  (TD)  phase  (MS  A), 
from  TD  to  System  Development  and  Demonstration  (SDD)  phase  (MS  B),  and  from  SDD  to 
Production  and  Deployment  (PD)  phase  (MS  C).  At  MS  A,  B  or  C  a  PM  must  show  a  certain 
level  of  knowledge  about  the  system  has  been  attained  before  being  allowed  to  proceed.  This  is 
normally  demonstrated  by  the  program  meeting  established  exit  criteria  for  one  acquisition  phase 
and  entrance  criteria  for  the  next.  These  exit  and  entrance  criteria  have  associated  with  them 
specific  documentation  requirements  that  describe  program/system  maturity  indicative  of  what 
has  been  established  as  a  best  practice  knowledge  level.  The  concept  is  that  by  attaining  this 
knowledge  level  a  program  will  be  on  the  path  to  success  upon  entering  the  next  phase  of  the 
acquisition  process[28]. 

MS  B  -  Transition  from  TD  to  SDD 

For  instance  at  MS  B  (which  is  arguably  the  most  important  decision  point  in  the 
acquisition  process  since  this  is  the  point  at  which  an  acquisition  program  is  established) [29]  a 
PM  for  an  Acquisition  Category  ID  (ACAT  1D)[30]  program  in  concert  with  the  Service  and 
OSD  staffs  must  produce  and  submit  up  to  as  many  as  35  documents  to  demonstrate  the  level  of 
knowledge  maturity — or  “business  case”  in  commercial  industry  parlance — for  the  program  to 
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leave  the  TD  phase  and  enter  SDD.  The  primary  focus  of  the  TD  phase  is  to  determine  and  then 
mature  the  critical  technologies  of  a  proposed  system  to  the  point  they  have  been  “demonstrated 
in  a  relevant  environment  and  a  system  (based  on  these  technologies)  can  be  developed  for 
production  within  a  short  timeframe  (normally  less  than  five  years)”.  The  establishment  and 
maturation  of  critical  technologies  is  a  collaborative  and  “iterative  process  designed  to  assess  the 
viability  of  technologies  while  simultaneously  refining  user  requirements”[311.  The  culmination 
of  the  TD  phase  is  a  match  between  mature  technologies,  user  requirements  and  funding  such 
that  a  specific  spiral  or  increment  of  system  capability  can  be  developed  and  provided  to  the 
warfighter  quickly  once  the  program  has  attained  a  successful  MS  B  decision  and  entered  into 
SDD. 

To  this  end  and  in  keeping  with  policies  of  Flexibility,  Responsiveness,  Innovation, 
Discipline,  and  Streamlined  and  Effective  Management  defined  in  DoDD  5000.1,  the  MDA 
along  with  the  PM  shall  establish  the  structure,  practices  and  documentation  that  will  be  used  to 
“acquire  quality  products  that  satisfy  user  needs  with  measurable  improvements  to  mission 
capability  and  operational  support,  in  a  timely  manner,  and  at  a  fair  and  reasonable  price.”  Table 
E3.T1  of  Enclosure  3  to  DoDI  5000.2  lists  the  35  key  documents  categorized  as  either  Statutory 
or  Regulatory  that  have  been  determined  to  provide  the  minimum  level  of  program  information 
necessary  make  a  decision  on  the  “business  case”  at  MS  B  to  proceed  from  TD  to  SDD 
phase[32].  This  list  is  supplemented  by  the  Defense  Acquisition  Guidebook,  which  provides  an 
online  set  of  references  covering  “non-mandatory  guidance  on  best  practices,  lessons  learned  and 
expectations”[33]. 

The  Technology  Readiness  Assessment  and  accompanying  Independent  Technology 
Assessment  are  examples  of  key  regulatory  documents  used  to  determine  the  program’s  critical 
technology  maturity  and  readiness  to  proceed  into  product  development.  The  Defense 
Acquisition  Guidebook  and  Technology  Readiness  Assessment  Deskbook  have  additional  best 
practice  and  expectation  reference  information  listed  for  these  documents.  In  the  Deskbook, 
extensive  treatment  is  given  to  the  explanation  of  technology  readiness  levels  used  to  define 
technology  maturity  and  the  responsibilities  of  both  the  PM  and  the  independent  technology 
assessment  team  responsible  for  providing  a  true  assessment  of  a  technology’s  maturity.  While 
the  DoD  policy  stipulates  that  technology  rated  at  a  TRL  of  6  or  less  can  be  brought  into  the 
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SDD  phase  if  required  and  with  appropriate  guarantees/waivers,  the  Deskbook  goes  to  great 
lengths  to  explain — using  GAO  citations  of  commercial  best  practices  and  numerous  DoD 
failures — why  a  TRL  7  (technology  at  a  prototype  maturity  level  that  has  been  demonstrated  in 
an  operational  environment)  or  better  should  always  be  used. 

Other  key  documents  in  the  DoDI  5000.2  table  listed  as  statutory  requirements  for  MS  B 
include  the  Acquisition  Program  Baseline,  Independent  Cost  and  Manpower  Estimates,  Selected 
Acquisition  Report,  Low  Rate  Initial  Production  Quantities,  and  the  Technology  Development 
Strategy.  As  with  the  Technology  Readiness  Assessment,  and  the  other  regulatory  documents,  all 
of  these  statutory  documents  have  amplifying  information  about  them  in  the  Defense  Acquisition 
Guidebook  that  provides  further  detail  about  what  the  document  is  for,  hyperlinks  to  the 
requiring  laws,  timeframes  for  their  submission,  and  when  available  a  link  to  further  detail  about 
best  practices  and  document  templates.  Along  with  the  amplifying  document  information  the 
Guidebook  provides  PMs  with  “with  discretionary  best  practices  that  should  be  tailored  to  the 
needs  of  each  program.”  This  information  is  broken  into  chapters  “designed  to  improve 
understanding  of  the  acquisition  process  and  ensure  adequate  knowledge  of  the  statutory  and 
regulatory  requirements  associated  with  the  process”[341. 

The  ACS  Program 

It  was  with  the  development  of  these  documents,  using  the  references  and  amplifying 
information  in  the  Defense  Acquisition  Guidebook,  that  the  Aerial  Common  Sensor  (ACS) 
program  proceeded  through  the  Defense  Acquisition  System  review  process  to  a  MS  B  decision. 
At  each  stage  of  the  review  process  details  of  the  documents  were  presented  to  the  decision 
authorities  who  based  on  this  information  considered  the  program  ready  to  proceed.  Because  of 
the  lack  of  major  issues  or  discussion  topics  concerning  the  program’s  readiness  to  proceed  to  a 
MS  B  Defense  Acquisition  Board  (DAB)  Review,  the  program  was  allowed  to  forego  the  formal 
DAB  meeting  and  instead  was  approved  for  transition  to  SDD  via  circulating  among  the  board 
members  for  their  approval  the  document/briefing  chart  package  that  would  have  been  presented 
had  the  meeting  taken  place.  With  the  approval  of  the  all  of  the  board  members  following  the 
addition  of  funding  to  the  program  required  by  the  Cost  Analysis  Improvement  Group,  the 
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program  was  approved  for  transition  into  SDD.  The  ACS  Acquisition  Decision  Memorandum 
was  published  on  29  July  2004[351. 

Why  Use  ACS  as  a  Case  Study? 

The  ACS  program  is  unique  in  that  it  is  an  AC  AT  ID  program  where  the  timeline  from 
pre-SDD  activity,  SDD  start,  and  SDD  contract  termination  occurred  in  a  three  year  period  and 
one  product  manager’s  tour  of  duty  with  all  senior  leaders  in  place  from  the  beginning  to  the  end 
of  this  time  period.  These  facts  provide  a  unique  situation  and  view  into  events  that  normally 
take  several  more  years  and  many  more  players  before  they  come  to  light  in  the  current 
acquisition  environment.  Beyond  this,  internal  and  external  teams  both  have  reviewed  the 
program’ s  events  that  led  to  its  contract  termination  during  the  final  months  of  the  contract’ s 
execution  and  immediately  following  contract  termination.  Capturing  the  accounts  of  what 
occurred  or  didn’t  as  seen  by  differing  points  of  view — while  the  events  are  still  relatively  fresh 
in  the  minds  of  those  that  participated  along  with  the  documents  supporting  the  program  still  in 
place  and  easily  reviewed — provides  a  unique  opportunity  to  determine  what  actually  happened 
and  to  gather  lessons  learned.  Also  of  note  is  that  while  the  ACS  SDD  contract  has  been 
terminated,  the  program  remains  in  post  MS  B  status  and  has  a  program  element  funding  line 
expecting  a  development  contract  restart  in  the  FY09  timeframe.  This  gives  the  members  of  this 
program  an  opportunity  of  a  second  chance  at  success  by  using  the  lessons  learned  form  the  first 
attempt.  Such  an  opportunity  is  something  rare  in  the  current  budget  environment. 

ACS  Program  Background 

The  ACS  system  is  designed  to  meet  the  Army  and  Navy’s  future  airborne  intelligence, 
surveillance,  and  reconnaissance  requirements.  It  is  intended  to  be  a  multi-intelligence  (MULTI- 
INT)  system  meaning  that  it  is  required  to  carry  not  only  a  signals  intelligence  (SIGINT) 
collection  capability,  but  also  imagery  intelligence  (IMINT),  and  a  measurements  and  signatures 
(MASINT)  collection  capability.  The  system  will  allow  the  retirement  of  both  the  Army’ s 
Guardrail/Common  Sensor  (GR/CS)  and  Airborne  Reconnaissance  Low  (ARL)  aircraft  systems 
as  well  as  the  Navy’s  EP-3E  aircraft  fleet.  To  meet  the  retirement  needs  of  theses  systems,  the 
Army  planned  for  the  ACS  initial  operational  capability  to  occur  in  2010.  The  ACS  program 
entered  System  Development  and  Demonstration  (SDD)  on  29  July  2004  based  on  the 
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acquisition  decision  memorandum  signed  out  by  the  Under  Secretary  of  Defense  for  Acquisition, 
Technology,  and  Logistics  (USD,  AT&L)  of  the  same  date.  Prior  to  this  date,  the  ACS  program 
had  successfully  completed  both  the  Concept  Exploration  (CE)  and  Technology  Development 
(TD)  phases  of  the  program  life-cycle.  In  2003,  as  the  Army  ACS  program  was  completing  its 
TD  phase,  the  Chief  of  Naval  Operations  (CNO)  directed  that  the  Navy  enter  into  a  partnership 
with  the  Army  for  the  ACS  procurement.  This  partnership  was  recognized  by  the  Army 
leadership  (Army  Acquisition  Executive  and  Vice  Chief  Staff  Army)  during  the  August  2003 
ASARC  meeting  held  to  confirm  that  the  ACS  program  successfully  met  the  exit  criteria  for  the 
TD  phase  and  was  prepared  to  enter  into  the  process  of  obtaining  a  MS  B  decision. 

On  20  October  2003,  the  Joint  Requirements  Oversight  Council  (JROC)  approved  the 
Army’s  Operational  Requirements  Document  (ORD).  While  the  Army’s  ORD  was  considered  by 
the  Navy  to  meet  approximately  98%  of  its  operational  requirements,  in  September  2003  the 
Army  agreed  to  add  two  additional  manned  workstations  to  the  baseline  aircraft  system 
configuration  to  accommodate  Navy  mission  needs.  The  remainder  of  Navy’s  requirements  were 
captured  in  the  Navy  ACS  ORD  Annex  that  was  approved  by  the  JROC  on  3  May  2004.  The 
Navy’s  plan  was  that  as  Navy  funding  became  available  they  would  add  the  additional  annex 
capability  to  the  ACS  system. 

The  ACS  program  had  now  evolved  in  the  last  year  before  the  MS  B  decision  from  an 
Army  only  ACAT  III  program  to  a  Major  Defense  Acquisition  Program  (ACAT  ID)  with  joint 
oversight  interest  and  the  Navy  considered  to  be  an  equal  partner.  Based  on  historical  ownership 
and  service  funding  levels,  the  agreement  was  that  the  Army  would  remain  the  lead  service.  For 
the  SDD  phase  of  the  program,  records  show  that  the  Army  was  expected  to  fund  $1.1  billion  of 
the  development  effort  with  the  Navy  funding  $170M.  The  concept  for  program  execution  was 
that  the  Army  had  contract  responsibility  and  program  management  responsibility  through  the 
Army  Acquisition  Executive,  and  the  Assistant  Secretary  of  Defense  for  Networks,  Intelligence 
and  Information  (ASD/NII)  to  Defense  Acquisition  Executive  as  the  Milestone  Decision 
Authority.  This  put  overall  program  execution  responsibility  on  the  shoulders  of  the  Army’s 
Lieutenant  Colonel  level  Product  Manager. 
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ACS:  What  Happened? 

In  September  2004,  one  month  after  contract  award,  the  Army,  Navy  and  contractor 
along  with  the  Defense  Acquisition  University  (DAU)  teams  met  for  a  post  award  conference. 
With  DAU  as  the  post  award  conference  facilitator — something  they  had  done  on  a  recent  Navy 
program  start-up — joint  vision  and  mission  statements  were  created,  and  Government 
(Army/Navy)/contractor  Integrated  Process  Teams  (IPT)  were  chartered  for  each  of  the 
functional  areas  of  system  development.  In  December  2004,  during  the  first  Kaizan  event  held  by 
the  aircraft  IPT,  it  became  apparent  that  the  estimated  weight  of  the  ACS  payload  was  100 
pounds  over  the  structural  limit  of  the  modified  commercial  aircraft  that  was  part  of  the 
contractor’s  system  design.  This  put  the  ACS  system  in  a  situation  where  the  aircraft  sub-system 
could  not  take-off. 

Starting  in  January  2005,  the  contractor  brought  into  the  program  weight  reduction 
experts  from  the  F-22  and  C-130-J  programs  to  find  areas  in  the  design  where  weight  could  be 
reduced.  To  accomplish  this,  they  accelerated  portions  of  the  design  to  gain  greater  fidelity  on 
the  exact  weight  drivers.  However,  instead  of  reducing  weight  as  the  design  matured,  the 
opposite  happened  and  the  expect  weight  of  the  system  design  continued  to  grow.  What  became 
apparent  as  the  system  design  matured  was  that  several  required  areas  of  the  payload 
infrastructure  had  either  been  overlooked  or  considerably  underestimated. 

One  area  of  underestimation,  which  became  a  focal  point  during  the  post-contract 
termination  Congressional  reviews  of  the  program,  was  the  total  cable  weight  required  to  connect 
the  numerous  signal  collecting  antennas  on  the  wings  and  fuselage  of  the  aircraft  to  the 
intelligence  processing  boxes  had  been  underestimated.  Another  area  of  “realized”  weight  during 
design  maturation  was  the  aircraft  structural  design  changes  made  to  attain  required  flight 
altitude  and  endurance.  While  the  flight  altitude  and  aircraft  endurance  were  non-Key 
Performance  Parameter  (KPP)  requirements  set  forth  in  the  ORD,  they  became  driving 
requirements  for  the  program  and  equivalent  to  KPPs  based  on  Army  and  Navy  user’s  detailing 
their  concepts  of  operation  (CONOPS)  for  the  system  during  the  Government’s  source  selection 
process  prior  to  contract  award.  This  made  it  almost  impossible  to  make  any  trades  to  reduce 
weight  in  this  area.  Other  areas  where  design  maturation  uncovered  additional  weight  were  the 
areas  affected  by  the  use  of  the  military  specifications  requiring  that  the  aircraft/payload  to  be 
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able  to  survive  a  16G  crash  versus  the  commercial  requirement  of  surviving  a  9G  crash.  The 
Government’s  Aircraft  Qualification  Plan,  which  delineated  aircraft/payload  survivability  “G” 
ratings  and  was  provided  to  the  contractor  during  source  selection,  had  been  provided  back  to  the 
Government  by  the  contractor  in  its  final  proposal  submission  with  a  change  reducing  the 
survivability  rating  from  16  to  9Gs.  The  16G  aircraft  survivability  requirement  called  out  in  the 
Government’s  version  of  this  document  was  still  under  negotiation  between  the  SDD  contractor 
and  the  Government  at  contract  termination. 

By  March  2005,  it  was  apparent  that  the  contractor’s  selected  aircraft  could  not  meet  the 
full  set  of  payload  requirements  due  to  the  payload/infrastructure  weight.  Reduced  payload, 
flight  altitude  and  endurance  capable  ACS  system  options  were  reviewed.  While  several  of  the 
options  were  found  to  be  KPP  compliant,  the  user  communities  of  both  the  Army  and  the  Navy 
found  all  options  to  be  unacceptable  based  on  overall  system  performance  not  meeting  the  needs 
of  the  service’s  CONOPS  being  developed  in  detail  concurrently.  At  this  point,  the  Army  and 
Navy  Program  Executive  Officers  (PEO)  determined  that  larger  payload  capable  aircraft  should 
be  reviewed  with  cost  and  schedule  implications  addressed  to  determine  if  viable  program 
options  were  available  within  the  assigned  Acquisition  Program  Baseline  (APB). 

The  ACS  PM  submitted  a  program  Schedule  Deviation  Report  (SDR)  in  May  2005  to  the 
MDA  to  inform  him  that  an  APB  schedule  breach  was  expected  on  the  ACS  program.  Even 
though  the  contractor  committed  to  and  began  tracking  earned  value  metrics  against  its  internal 
baseline  for  the  program  in  December  of  2004,  due  to  the  ability  of  the  Government/contractor 
team  to  close  on  a  design  that  provided  needed  system  capability,  the  Government  did  not 
accomplish  the  customary  Initial  Baseline  Review  (IBR)  to  validate  and  accept  the  contractor’s 
program  baseline.  Although  there  was  no  formal  Government  approved  program  baseline,  the 
PM  submitted  the  SDR  because  it  was  quite  evident,  even  from  the  non-validated  program 
earned  value  measurement  (EVM)  statistics,  that  the  weight  issue  the  program  was  experiencing 
and  the  resulting  effort  to  rectify  the  problem  was  keeping  the  contractor  from  completing 
expected  contracted  work  packages  during  the  same  timeframe.  This  meant  that  the  program’s 
major  milestone  events  such  as  the  System  Requirements  Review  and  Preliminary  Design 
Review  were  either  being  executed  unsatisfactorily  or  not  at  all. 
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The  Government/contractor  ACS  team  evaluated  several  larger  aircraft  based  program 
options.  All  options  that  were  expected  to  meet  the  ORD  and  CONOPS  performance 
requirements  breached  the  APB  in  both  cost  and  schedule.  In  July  2005,  these  higher  cost 
estimates  were  validated  by  the  external  non-advocacy  review  (NAR)  team  commissioned  by  the 
Army  PEO  to  review  the  program’s  viability.  The  NAR,  consisting  of  senior  acquisition 
experienced  personnel  from  both  the  Navy  and  Army,  conservatively  estimated  that  a  viable 
SDD  program  would  cost  between  $2.5B  -  $2.9B  depending  on  whether  the  estimate  was  for  an 
Army  only  program  or  included  Navy  test  assets  respectively.  This  was  over  100%  more  than  the 
APB  expected  SDD  cost  of  $1.2B.  Interestingly,  during  this  program  option  review,  the  baseline 
ACS  SDD  program  was  also  redressed.  The  Government/contractor  team  found  that  instead  of 
the  baseline  ACS  development  effort  costing  the  contracted  $820M  it  was  now  estimated  to  cost 
$1.1B  for  the  contractor  to  develop  a  system  that  by  this  time  had  been  determined  could  not 
take-off  (1.  IB  is  a  contractor  only  cost  which  does  not  include  the  estimated  $100M  for 
Government  program  office  operating  costs  for  the  length  of  the  development  effort). 

On  September  14,  2005,  the  Army  issued  a  Stop  Work  Order  (SWO)  to  the  contractor  for 
all  contracted  efforts.  This  decision  was  made  following  an  emergency  ASARC  that  was  called 
by  the  Army  Acquisition  Executive  to  review  a  recommendation  for  contract  termination  made 
on  8  September  2005.  The  SWO  called  for  the  contractor  to  focus  efforts  on  “alternate  strategies 
for  Army  consideration  that  maximize  possible  performance  while  minimizing  negative  cost  and 
schedule  impact  to  the  Government” [3 61.  With  this  direction,  the  contractor  was  provided  the 
opportunity  to  take  sixty  days  to  develop  a  “written  plan”  that  demonstrated  their  best  effort  to 
meet  the  needs  of  the  user  while  minimizing  cost  and  schedule.  The  contractor  provided  three 
system  options  based  on  three  different  aircraft.  While  two  of  the  aircraft  system  solutions  met  or 
exceeded  KPPs  and  the  critical  driving  threshold  requirements  of  altitude  and  endurance,  the  cost 
and  schedule  of  each  breached  the  APB .  The  third  aircraft  solution  was  based  on  the  original 
planned  contractor  aircraft  and  design  concept.  While  this  aircraft  based  design  met  the  ORD 
KPPs,  it  required  the  user  to  accept  a  significantly  reduced  capability  in  the  driving  requirements 
and  would  only  accommodate  four  instead  of  six  operators  required  by  the  Navy.  This  alternative 
also  breached  the  APB  cost  and  schedule. 
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After  an  extension  to  the  SWO  to  allow  the  contractor  more  time  to  review  alternate 
available  technologies  to  reduce  weight  ended  without  success,  the  contract  was  terminated  on 
12  January  2006[37].  The  contract  was  terminated  for  several  reasons.  First,  the  cost  of  each 
alternate  aircraft  course  of  action  would  require  significantly  more  money  and  time  to  execute 
based  on  the  estimates  provided.  Executing  such  a  course  of  action  without  the  benefit  of 
competition  was  seen  as  unwise  due  to  the  likelihood  that  other  competitors  would  protest.  This 
is  especially  true  if  one  considers  that  a  critical  factor  during  the  source  selection,  which 
significantly  impacted  the  Source  Selection  Authority’s  decision  to  select  the  ACS  contractor, 
was  based  on  the  aircraft’s  ability  to  meet  the  requirements  of  payload  weight,  aircraft  altitude 
and  aircraft  endurance.  Second,  during  the  August  2004  ASARC  leading  to  the  MS  B  decision, 
the  PM  had  established  metrics  that  if  triggered  would  be  cause  for  contract  termination.  Those 
were:  “Failure  to  support  critical  milestones  and  failure  to  support  KPPs”[38].  Considering  the 
state  of  the  contractor’s  performance  due  to  the  payload  weight  issue,  critical  milestones  were 
not  being  met.  And,  while  it  was  determined  through  analysis  that  the  new  system  designs 
offered  by  the  contractor  could  theoretically  meet  all  KPPs,  the  NAR  had  expressed  significant 
concern  over  the  new  technology  development  path  being  taken  by  the  contractor  on  a  critical 
portion  of  the  IMINT  subsystem  package.  Third,  the  Government  program  management  team 
after  having  worked  with  the  contractor  to  solve  the  weight  issue  was  very  concerned  about  the 
contractor’s  technical  ability  to  execute  the  development  effort.  Through  the  many  reviews  of  the 
courses  of  action  presented  by  the  contractor,  it  was  apparent  that  the  new  contractor  team 
members — both  in  leadership  and  key  team  member  positions — brought  on  after  SDD  contract 
award  were  on  a  steep  learning  curve  as  to  what  the  ACS  system  was  to  provide  and  how  best  to 
develop  the  design. 

Army/Navy  Government  Team  Relationship 

The  Government  program  management  team  relationship  between  the  Army  and  Navy 
was  tenuous  and  at  times  even  hostile  during  the  execution  of  the  source  selection  process  and 
the  SDD  contract.  While  there  were  IPT’s  in  which  the  Army  and  Navy  co-leads  complemented 
each  other  well  and  daily  activities  focused  on  the  business  of  managing  their  functional  area, 
more  times  than  not,  the  two  service  co-leads  struggled  with  how  to  approach  IPT  management 
and  interpretation  of  related  program  requirements.  This  struggle  stems  from  two  diametrically 
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opposed  views  of  acquisition  strategy.  The  Army’s  program  management  view  of  the  acquisition 
strategy  for  the  ACS  program  can  be  found  in  the  ACS  ORD  and  Acquisition  Strategy 
documents  where  it  states  that  cost  will  be  used  as  an  independent  variable.  Cost  As  an 
Independent  Variable  (CAIV)  was  the  defining  concept  on  which  the  initial  program  schedule  of 
a  five-year  SDD  phase  was  based.  Of  the  divine  triad  of  cost,  schedule  and  performance,  this 
meant  that  the  Army  expected  that  non-KPP  performance  would  be  flexed  to  maintain  SDD  cost 
and  schedule.  The  Navy’s  program  management  view  was  based  on  meeting  performance 
requirements.  This  was  expressed  by  the  Navy’s  PEO  as  a  culture  of  NAVAIR.  While  meeting 
cost  and  schedule  is  important  to  the  Navy,  meeting  all  user  stated  performance  requirements 
was  paramount.  Also,  the  Navy  ACS  program  representatives  believed  that  they  had  already 
traded  as  many  Navy  capability  requirements  as  they  believed  reasonable  to  enable  them  to 
consider  the  ACS  system  a  viable  candidate  to  replace  the  EP-3E.  However,  this  understanding 
did  not  come  to  light  until  after  contract  award  and  the  Government/contractor  teams  were 
working  on  acceptable  solutions  for  the  system  payload  weight  issue.  Another  cause  of  this 
opposing  acquisition  philosophy  view  can  perhaps  be  linked,  to  a  degree,  back  to  the 
apportionment  of  the  DoD  budget  and  the  differing  service  cultures  that  develop  because  of  it. 
While  some  might  say  this  a  stretch,  what  cannot  be  denied  is  that  the  Navy  does  receive  a  larger 
portion  of  the  DoD  investment  account  budget  than  the  Army.  This  might  allow  them  a  mind-set 
of  available  budget  not  being  as  great  an  issue  as  it  is  with  the  Army.  In  FY07  for  example,  the 
Army  received  14.8%  while  the  Navy  received  23.3%  of  the  investment  account  budget[39]. 
Interestingly,  because  of  the  Army’s  lack  of  additional  budget  authority  to  cover  the  Cost 
Analysis  Improvement  Group’s  (CAIG)  increased  SDD  program  cost  estimate  developed  in 
preparation  for  the  MS  B  decision,  the  Navy  was  persuaded  by  the  Army  to  “buy-into”  the 
program  and  provide  the  need  $170M  across  the  POM  to  meet  the  CAIG  SDD  cost  estimate. 

An  additional  factor  that  played  into  the  Army/Navy  relationship  was  the  difference  in 
management  practices  and  team  locations.  The  Navy  side  of  the  Government  team  while  as  large 
as  the  Army  had  a  distinct  difference  in  composition.  While  the  Army  used  a  mixture  of  support 
contractors  and  Government  civilians  who  were  technical  experts  in  their  area  of  expertise  as  IPT 
leads,  the  Navy  ensured  that  all  co-IPT  leads  were  Government  civilians.  The  Navy  believed  that 
this  was  necessary  to  ensure  that  each  of  the  IPT  leads  could  speak  on  behalf  of  the  Government 
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in  the  execution  of  their  chartered  IPT  responsibilities.  This  is  necessary  because  of  the  way  in 
which  the  Navy  (NAVAIR)  manages  the  contractor  efforts.  Specifically,  the  Navy  normally  pairs 
its  Government  IPT  lead  with  the  contractor  IPT  lead  and  makes  the  Government  civilian  equally 
responsible  through  their  performance  appraisal  for  the  cost,  schedule  and  performance  of  the 
subsystem  component  development  for  which  their  IPT  is  responsible. 

On  the  other  hand,  the  Army’s  approach  was  that  the  contractor  IPT  lead  was  responsible 
for  the  subsystem  development  and  its  cost,  schedule  and  performance  for  which  they  had 
responsibility  based  on  contractor  allocated  work  breakdown  structure.  The  role  of  the 
Government  IPT  co-lead  was  to  be  the  eyes  and  ears  of  the  Government  PM  and  to  facilitate  the 
communication  exchange  between  the  contractor  and  the  Government  to  ensure  that  needed 
information/approvals  and  required  documentation  from  the  Government  for  the  specific  IPT 
was  provided  in  the  most  expeditious  manner.  The  Government  IPT  co-lead  could  not  speak  on 
behalf  of  the  PM  to  make  changes  to  work  efforts  unless  expressed  permission  was  given  first. 

Additionally,  the  Army  and  Navy  Government  SDD  team  members  were  located  in 
different  locations  separated  by  an  approximately  five-hour  drive.  This  significantly  hampered 
face-to-face  communications  and  promoted  suspicion  and  the  belief  by  the  Navy  that  their  voice 
was  not  being  heard.  The  Navy  also  believed  that  a  development  program  that  included  an 
aircraft — even  if  it  included  a  non-developmental  modified  commercial  aircraft — should  be 
managed  from  an  organization  that  focused  on  developing  and  deploying  aircraft  systems — 
specifically  NAVAIR  from  whence  the  Navy  team  came.  This  played  significantly  in 
contributing  to  the  tenuous  and  sometimes  hostile  nature  of  the  relationship  especially  when 
combined  with  the  differing  management  beliefs  discussed  above  and  the  duality  of  the  two 
internal  Government  management  structures.  Essentially,  the  ACS  program  had  two  separate 
program  management  teams  with  completely  separate  reporting  chains.  While  the  Army  PM,  an 
05,  had  reporting  responsibility  through  the  Army  06  PM  to  the  Army  PEO,  and  AAE;  the  Navy 
PM,  a  GS-15,  had  reporting  responsibility  to  the  Navy  06  PM,  who  reported  to  the  Navy  PEO, 
who  reported  to  the  NAE  on  ACS  program  progress. 

Part  of  this  duality  might  be  attributed  to  the  fact  that  the  Navy  never  fully  joined  the 
ACS  program  as  directed  by  ACS  Acquisition  Decision  Memorandum  signed  out  by  the  DAE  on 
29  July  2004.  The  ADM  called  for  the  Navy  to  return  to  the  DAE  by  December  2004  once  the 
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Navy’s  full  set  of  requirements  had  been  establish,  funding  obtained  and  the  combined 
Army/Navy  APB  had  been  established  and  signed  out  by  the  respective  services.  However,  due 
the  system  design  weight  issues  which  caused  the  system  development  schedule  to  slip  and  an 
Initial  Baseline  Review  not  to  be  conducted,  the  Navy  could  not  and  would  not  commit  to  a  cost, 
schedule  and  performance  Acquisition  Program  Baseline.  So,  while  the  Navy  was  considered  a 
full  partner  and  had  participated  in  the  source  selection  process  as  members  of  the  source 
selection  evaluation  board,  the  source  selection  advisory  committee  and  in  the  in  day  to  day 
management  of  the  ACS  SDD  development  program,  they  were  not  fully  committed  to  meeting 
the  program  schedule  or  cost  metrics  as  established  by  the  Army  APB  under  which  the  program 
was  operating.  Combine  this  with  the  skepticism  of  the  Army  management  approach  and  use  of 
CAIV  to  meet  a  five  year  development  schedule  to  deliver  a  first  ACS  unit  to  the  user  by  2010, 
and  you  have  a  Government  team  that  was  internally  divided. 

Pre-System  Development  and  Demonstration 

To  fully  understand  how  and  why  the  events  unfolded  during  the  SDD  phase  discussed 
above  which  led  to  contract  termination,  it  is  important  to  look  back  at  the  TD  phase  and  the 
period  of  time  between  the  end  of  TD  and  the  SDD  contract  award.  By  doing  this,  several  more 
of  the  pitfalls  that  the  program  faced  utilizing  the  current  acquisition  system  begin  to  take  shape. 
Recognition  of  these  pitfalls  is  the  first  step  in  identifying  the  lessons  to  be  learned  that  may 
ultimately  to  become  candidates  that  can  be  used  to  identify  the  “why”  of  best  practices  that  may 
be  adopted  in  the  future. 

The  competitors  for  the  SDD  contract  had  been  narrowed  to  from  three  to  two  contractors 
between  the  end  of  the  18 -month  long  CE  phase  and  the  start  of  the  TD  phase  of  the  acquisition 
process.  Of  the  two  contractors  who  continued  though  TD,  one  was  considered  the  incumbent  as 
they  had  developed  and  fielded  the  Army’s  current  GR/CS  airborne  SIGINT  system  which  ACS 
was  intended  to  replace.  The  competing  contractor,  while  not  having  the  same  experience,  had 
developed  and  published  a  novel  open  architecture  approach  for  integrating  subsystems  into  the 
ACS  system.  The  ACS  team  considered  this  approach  to  have  the  potential  for  providing  ease  of 
subsystem  integration  and  possible  competition  for  capability  upgrades  in  the  future.  As  with  the 
incumbent,  they  too  by  the  end  of  TD  appeared  to  have  a  good  understanding  of  the  user’s  needs. 
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The  ACS  TD  phase  was  a  15-month  effort  started  in  April  2002  and  completed  in  July 
2003.  The  program  entered  TD  to  further  develop  the  concepts  that  were  drafted  during  CE  and 
to  mitigate  the  development  risk  of  critical  technologies  for  the  system  development.  A  critical 
technology  at  risk  was  the  SIGINT  subsystem  due  to  the  cancellation  of  the  Joint  Low  Band 
Subsystem  (LBSS)  program.  LBSS  was  an  Air  Force  led  joint  program  expected  to  deliver  a 
common  airborne  SIGINT  technology  subsystem  that  would  be  used  by  all  three  services  in  the 
development  of  their  service  unique  airborne  SIGINT  systems.  Considering  the  low  density  of 
airborne  signals  intelligence  systems  and  the  cost  of  developing  SIGINT  subsystems,  this 
concept  of  developing  a  subsystem  once  and  having  it  used  by  all  made  economic  and 
interoperability  sense.  One  of  the  reasons  for  its  cancellation  was  that  the  increase  in 
requirements  or  “requirements  creep”  from  the  three  services  was  making  the  subsystem  too 
complex  and  requirements  too  unstable  to  develop  to.  Without  this  SIGINT  subsystem,  the  ACS 
program  would  have  to  develop  its  own  capability  during  SDD.  Demonstrating  SIGINT  design 
approaches  during  TD  was  a  primary  focus.  Other  risk  areas  addressed  during  TD  were  system’s 
architecture  and  integration  and  multi-INT  man  machine  interface. 

The  ACS  program  sustained  a  funding  cut  during  TD  in  the  aftermath  of  the  LBSS 
program  cancellation.  While  the  Government  attempted  to  limit  the  impact  of  the  cut  on  contract 
execution,  they  were  unable  to  fully  mitigate  the  cut  without  de-scoping  a  portion  of  the 
contracted  work  effort.  Because  of  the  risk  in  the  SIGINT  technology  area  left  by  the 
cancellation  of  the  LBSS  contract,  the  decision  was  made  to  limit  the  extent  that  contractors  were 
required  to  demonstrate  ACS  system  level  integration.  The  assumption  was  that  each  contractor 
had  significant  experience  in  integrating  payloads  on  an  aircraft  based  on  their  previous  contracts 
for  airborne  system  developments. 

By  the  end  of  TD,  each  contractor  had  demonstrated  technology  risk  area  elements  of 
their  approaches,  and  showed  consideration  for  “SIGINT  System  development  (demonstrate 
feasibility  of  the  SIGINT  solutions  and  include  a  plan  for  the  migration  of  the  objective  SIGINT 
subsystem);  platform  integration  (i.e.  antenna,  subsystems);  Multi-Node/Multi-INT  system 
complexity,  distribution  of  system  and  data  across  multiple  ground  and  airborne  elements;  and 
Human  Machine  Interface” (40]  as  required  by  the  Milestone  Decision  Authority.  These 
considerations  were  also  reflected  in  the  five  TD  exit  criteria  that  were  established  as  the  metrics 
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to  determine  if  the  program  had  matured  the  needed  technologies  to  proceed  into  SDD.  The  MS 
B  decision  authority  considered  that  all  exit  criteria  had  been  met. 

Additionally,  during  TD  each  contractor  provided  the  Government  with  a  non¬ 
proprietary  draft  Performance  Based  Specification  (PBS).  This  PBS  was  based  on  a  draft  ORD 
furnished  them  by  the  Government  early  in  TD.  The  Government  later  used  the  best  from  each  of 
the  draft  PBS’  along  with  the  JROC  approved  ORD  and  personal  experience  with  the  current 
airborne  SIGINT  systems  to  create  the  Government  PBS  that  was  sent  to  each  contractor  team  as 
part  of  the  December  2003  solicitation  package  for  the  SDD  contract. 

Between  the  end  of  the  TD  phase  and  the  ACS  SDD  contract  award  there  was  a  14-month 
period  in  which  the  Government  accomplished  its  MS  B  preparation  and  source  selection 
process.  During  this  period  Government/contractor  interaction  was  minimal.  With  the  end  of  the 
TD  contract  in  June  2003,  outside  of  the  contractor  responding  to  the  Request  for  Proposal  (RFP) 
there  was  little  formal  discussion  between  the  contractor  and  the  Government.  However,  during 
this  period  many  events  were  occurring  that  would  have  an  impact  on  ACS  system  design 
ultimately  proposed  by  the  contractors  for  SDD  development. 

In  March  2003,  just  prior  to  the  TD  phase  ending,  the  Army  Requirements  Oversight 
Council  (AROC)  approved  the  ACS  ORD.  While  the  AROC  ORD  was  provided  to  the 
contractors  immediately  after  its  approval,  it  contained  significant  changes  from  the  draft  ORD 
that  the  contractors  had  been  using  to  develop  system  feature  to  that  point.  Many  of  these 
changes  were  in  the  critical  technology  areas  that  the  TD  phase  was  designed  to  focus  on. 
Specifically  the  AROC  ORD  required:  an  increase  in  the  flight  altitude  of  the  aircraft,  the  need 
for  all  ACS  aircraft  to  carry  and  operate  SIGINT,  IMINT  and  Satellite  Communications 
(SATCOM)  subsystems,  the  ability  to  self-deploy  within  72  hours  and  accomplish  autonomous 
operations  for  72  hours,  an  increase  from  two  operators  with  portable  workstations  to  four 
operators  with  fixed  on-board  workstations — the  additional  two  operators  accomplishing  the  new 
requirement  of  Battle  Command,  an  increase  in  the  number  of  other  systems  that  the  ACS  system 
would  have  to  exchange  data  with,  the  implementation  of  a  new  OSD  mandated  reliability  KPP, 
the  implementation  of  specified  performance  numbers  for  the  IMINT  KPP,  the  extension  of  the 
SIGINT  range  and  KPP  targets,  and  the  development  of  a  tri-band  SATCOM. 
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In  addition  to  these  changes,  four  additional  requirements  changes  took  place  between  the 
March  approved  AROC  and  October  approved  JROC  ACS  ORDs:  The  now  KPP  data  exchange 
matrix  was  expanded  again,  a  limitation  was  placed  on  the  length  and  type  of  airfield  that  aircraft 
had  to  land  on,  the  SIGINT  operational  frequency  range  was  expanded,  and  lastly  a  new 
requirement  was  levied  for  the  use  of  a  specific  type  of  Global  Positioning  System. 

Following  the  JROC  approved  ORD,  several  derived  and  specified  requirements  of 
similar  scope  and  focus  as  those  mentioned  above  were  placed  in  the  Government  Performance 
Based  Specification  issued  to  the  contractors  as  part  of  the  RFP  in  December  2003.  For  example, 
the  requirements  included  the  OSD  mandated  use  of  Internet  Protocol  version  6,  the  need  for 
16G  crash  survivability,  and  a  specified  data  throughput  rate  for  the  SATCOM.  This  SATCOM 
throughput  rate  significantly  increased  the  size  and  shape  of  the  fuselage  mounted  SATCOM 
radome.  Also,  it  was  during  this  period  that  Army  accepted  the  Navy’s  need  to  add  two  operator 
workstations  to  the  system  baseline  bringing  the  total  number  of  on-board  fixed  workstations  to 
six. 

While  the  number  and  type  of  requirements  changes  are  significant,  a  independent 
Technology  Readiness  Assessment  (TRA)  conducted  by  Deputy  Assistant  Secretary  of  Defense 
for  Research  and  Technology  (DASA(R&T))  accomplished  during  this  timeframe  found  that 
“the  risk  mitigation  approaches  presented  by  PM  ACS  were  deemed  sufficient  to  provide  timely 
increase  in  maturity  to  support  integration  into  ACS  during  SDD”[41].  Of  the  four  critical 
technologies  cited  in  the  TRA,  two  of  the  four  were  found  to  be  at  a  TRL  6  or  higher.  One 
technology  was  found  to  be  between  TRL  5  and  6  “with  a  viable  plan  to  get  to  TRL  6  early  in 
SDD.”  And,  the  final  technology  was  found  to  be  a  TRL  5  again  “with  a  viable  plan  to  get  to 
TRL  6  by  early  SDD.”  TRA  deskbook  guidance  states,  “that  although  there  is  no  rigid 
requirement  that  every  critical  technology  has  to  be  at  a  pre-specified  TRL  by  Milestone  B,  a 
level  6  is  typical  and  a  level  7  is  preferred” [42].  Despite  the  reference  in  the  TRA  Deskbook, 
based  on  the  DASA(R&T)  assessment  of  the  technology  maturity  of  the  ACS  critical 
technologies,  both  the  members  of  the  ASARC  and  later  the  DAB  found  the  ACS  program 
prepared  to  enter  into  SDD. 

In  July  2003  ACS  was  designated  a  Major  Defense  Acquisition  ACAT  ID  Program. 
Based  on  this,  the  program  was  required  to  have  an  independent  cost  estimate  (ICE) 
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accomplished  by  the  OSD  CAIG  prior  to  going  before  the  DAB.  Due  to  the  complexity  of  the 
types  of  technologies  being  used  on  the  ACS  program  along  with  the  recent  termination  of  the 
LBSS  program,  the  CAIG  wanted  the  opportunity  to  speak  to  the  contractors  directly  and  to 
review  the  contractor  proposals  before  finalizing  their  ICE.  “This  process  took  approximately  six 
months  with  the  final  estimated  CAIG  program  cost  being  taken  into  consideration  during  the 
source  selection  and  adding  to  the  target  cost  provided  to  each  contractor  for  their  Best  and  Final 
Offers  (BAFO)”[43].  Also,  because  of  the  length  of  time  that  the  source  selection  process  took 
including  the  time  to  work  with  the  CAIG  on  their  ICE,  the  PM  requested  and  was  granted  a  six 
month  extension  to  the  SDD  schedule  by  the  AAE  prior  to  presenting  APB  to  the  DAB.  This 
timeline  was  also  provided  to  the  contractors  to  consider  in  the  final  phase  of  the  source  selection 
process  as  they  submitted  their  BAFOs. 

On  29  July  2004,  the  Army  received  MS  B  approval  to  proceed  into  the  SDD  phase.  This 
is  based  on  the  fact  the  program  manager  demonstrated  to  the  MDA  that  he  had  satisfactory 
developed  all  of  the  required  DoDI  5000.2  documentation  for  an  ACAT  ID  program  and  that  the 
Army  had  established  a  funding  profile  to  meet  the  CAIG  ICE  SDD  program  cost.  The  PM  for 
ACS  developed  and  presented  all  required  documents  for  the  program  including  the  Acquisition 
Strategy,  Technology  Readiness  Assessment  (TRA),  and  the  System  Engineering  Plan  (SEP)  all 
of  which  were  reviewed  and  approved  by  the  appropriate  authority.  These  are  considered  the  key 
documents  that  layout  strategy  and  technology  readiness  so  often  touted  as  the  areas  normally  at 
the  root  of  program  failures.  Additionally,  these  documents  demonstrate  that  as  the  ACS 
program  proceeded  from  TD  to  the  MS  B  decision,  the  required  documentation  process  was 
followed  causing  outside  organizations  to  review  aspects  of  the  program’s  preparedness  to 
proceed  to  a  MS  B  decision  as  called  for  by  the  current  DoDD  5000  Acquisition  Process. 
Specifically,  the  TRA  and  the  SEP  received  multiple  reviews  before  final  approval  with  the  final 
version  of  the  SEP  (completed  after  contract  award)  being  touted  as  an  example  by  AT&L 
System  Engineering  that  would  be  given  to  future  programs  as  a  model  for  the  their  SEP 
developments.  Therefore,  as  the  ACS  program  proceeded  through  the  ASARC,  Integrated 
Integrating  Process  Team  (IIPT)  and  Overarching  Integrated  Process  Team  (OIPT)  reviews  the 
organizations  responsible  to  review  the  program  documentation  and  appraise  the  review  process 
leadership  on  program  readiness,  all  reported  that  the  program  had  satisfactorily  accomplished 
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the  necessary  work  to  proceed  to  the  Defense  Acquisition  Board  for  final  approval  to  move  into 
the  SDD  phase.  Of  note  is  the  fact  that  the  ACS  program  was  considered  to  have  no  major  issues 
associated  with  it  and  therefore  did  not  need  to  actually  go  before  the  DAB  in  the  form  of  a 
singular  meeting.  Instead  the  program  DAB  briefing  charts  and  required  documentation  package 
was  circulated  through  all  of  the  principle  DAB  member  organizations  for  their  approval  or 
disapproval  for  entering  into  MS  B.  Based  on  the  fact  that  this  gave  members  of  these 
organizations  a  degree  of  anonymity  to  more  readily  disapprove  the  package  if  they  felt  their  was 
an  issue  that  needed  to  be  addressed,  the  fact  that  no  organization  disapproved  of  the  program 
proceeding  into  SDD  would  lead  one  to  believe  that  it  truly  was  prepared  to  execute  a  successful 
SDD  program. 

ACS  Pitfalls  &  Lessons  Learned 

The  question  to  be  asked  is:  Based  on  the  events  of  the  ACS  Program  along  with  the 
understanding  of  the  current  DoDI  5000.2  Acquisition  Process  documents  for  ACAT  ID 
programs  as  they  proceed  from  TD  and  into  SDD  are  there  recognized  pitfalls  and  lessons  that 
can  be  learned  that  might  be  applied  by  future  PMs  to  avoid  similar  pitfalls?  The  answer  is  an 
obvious  “yes”  as  one  may  have  already  concluded  having  read  the  events  of  the  ACS  program 
described  above.  Additionally,  while  the  ACS  program  is  unique  in  its  execution  timeframe  for 
the  ACAT  ID  SDD  contract — only  one  year  and  one  month  before  the  Stop  Work  Order  was 
issued — the  pitfalls  faced  by  the  program  are  not  unique  to  the  ACS  program  but  have  been 
recognized  in  numerous  program  reviews  accomplished  by  GAO,  RAND  and  more  recently  by 
the  Under  Secretary  of  Defense  in  the  Defense  Acquisition  Performance  Assessment  (DAP A) 
report. 

Using  the  lens  of  the  ACS  program  provides  the  opportunity  to  not  only  to  recognize  the 
pitfalls,  but  also  to  focus  on  the  timeframe  and  context  in  which  they  occurred.  The  ACS 
program  specifically  brings  to  light  those  potential  pitfalls  that  are  lurking  in  the  Acquisition 
Process  during  the  TD  through  the  beginning  of  the  SDD  phases  of  a  Joint  ACAT  ID  program. 
Considering  that  this  is  a  critical  if  not  the  most  critical  time  in  the  life-cycle  of  an  acquisition 
program  because  it  sets  the  metrics  by  which  program  success  will  be  measured,  understanding 
the  pitfalls  and  possibly  avoiding  them  in  the  future  has  the  potential  to  provide  the  Acquisition 
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Community  the  opportunity  to  meet  cost,  schedule  and  performance  and  regain  the  confidence  of 
DoD  leadership  and  Congress. 

Pitfalls/Lessoned  Learned 

One  of  the  most  often  occurring  themes  of  an  AC  AT  ID  program  that  falters  during  SDD 
is  that  ultimately  the  available  budget  and  user  requirements  are  mismatched.  Causes  for  this 
mismatch  are  many  but  may  be  binned  into  these  categories:  budget  cuts,  requirements  changes 
after  funding  is  locked,  or  lack  of  well  defined  requirements  by  which  the  program’s 
development  cost  was  estimated.  The  ACS  program  suffered  all  of  these  causes  from  TD  through 
the  beginning  of  SDD.  These  causes  come  to  light  in  many  of  the  pitfalls  faced  by  the  program 
during  this  period [44]. 

Pitfall/Lesson  Learned  -  Funding  Process  versus  Requirements  Process:  Requirements 
Changes  after  Funding  Locked 

The  budget  process  and  the  requirements  validation  process  are  not  in  sync.  The  budget 
process  requires  program  funding  to  be  locked  at  least  two  years  before  actual  execution.  This 
occurs  while  the  Joint  Requirements  Oversight  Council  (JROC)  now  the  Joint  Capabilities 
Integration  and  Development  System  (JCIDS)  process  continues  to  develop  and  approve 
requirements  documents  out  of  sync  with  the  budget  timelines.  For  the  ACS  program,  the  fact 
that  significant  requirement  changes  occurred  at  the  end  of  TD  and  then  again  between  TD  and 
SDD  meant  that  the  budget  for  the  start  of  the  program  had  already  been  locked  and  did  not 
include  the  cost  impact  of  the  requirements  increase.  While  the  CAIG  did  require  that  additional 
funding  be  added  to  the  program,  because  of  the  budget  process,  funding  could  not  be  added  to 
the  early  years  of  the  SDD  program  when  it  would  have  had  the  greatest  impact. 

Pitfall/Lesson  Learned  -  Lack  of  Detailed  CON OPS: 

Lack  of  Well  Defined  Requirements 

Interpretation  and  understanding  of  requirements  does  not  only  come  from  the  ORD  or 
now  the  Capabilities  Development  Document  (CDD).  Another  source  is  the  CONOPS  which 
basically  defines  how  the  system  will  be  used.  Therefore,  a  detailed  CONOPS  document  is 
needed  early  in  the  acquisition  budget  process  to  ensure  that  the  requirements  are  well  defined 
and  understood.  Not  only  will  this  allow  for  accurate  program  costing,  but  also  help  to  avoid 
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requirements  “creep”  from  occurring  later  in  the  program.  The  ACS  program  did  not  have  a 
detailed  CONOPS  early  in  the  program.  There  was  a  top  level  discussion  of  CONOPS  in  the 
ORD,  however,  due  to  the  complexity  of  the  program  and  the  use  of  Commercial  Off  The  Shelf 
(COTS)  products  the  lack  of  a  detailed  description  of  how  the  system  would  be  used  left  several 
requirements  “hidden”  until  after  the  program  funding  had  been  locked.  For  instance,  once  the 
Navy  joined  the  program  and  developed  their  detailed  CONOPS  it  was  apparent  that  they 
planned  to  fly  several  types  of  IMINT  missions  at  greater  frequency  and  lower  altitude  than  the 
Army.  With  this  information,  the  Government/contractor  SDD  program  team  realized  after 
contract  award  that  to  meet  the  20-year  system  life-span  requirement,  additional  structure  would 
have  to  be  added  to  the  commercial  aircraft  wing.  This  was  not  previously  planned  for  nor  costed 
in  the  contractor’s  system  design. 

Another  example  of  CONOPS  impact  was  the  fact  that  ACS  system  was  a  “system  of 
systems”.  Specifically,  it  was  the  airborne  sensor  component  of  a  system  that  had  a  ground 
station  component — being  developed  by  a  separate  PM — that  collected  the  airborne  sensor  data 
and  then  distributed  it  as  required.  Once  the  detailed  CONOPS  was  developed,  it  was  apparent 
that  in  some  instances  the  Army  intended  to  deploy  ACS  ahead  of  the  ground  station.  The  team 
realized  that  this  would  require  a  method  for  the  deployed  ACS  system  to  “off-load”  and  secure 
its  collected  data  as  well  as  have  a  place  from  which  to  stage  and  execute  aircraft  and  subsystem 
maintenance  until  such  time  that  the  ground  station  deployed  if  it  did  deploy.  Again,  this  was  an 
unplanned  requirement  and  cost  that  was  not  part  of  the  contractor’s  original  system  design  at 
contract  award. 

The  final  example  of  the  need  for  a  detailed  ACS  CONOPS  early  in  the  acquisition  cycle 
was  because  of  its  significant  Space,  Weight,  Power  and  Cooling  (SWAP-C)  impact  on  the 
system  design.  Soon  after  contract  award,  the  Government/contractor  program  management  team 
was  told  by  the  user  representative  that  all  the  subsystems  had  to  operate  simultaneously  and  at  a 
lower  altitude  than  had  been  previously  thought.  This  “simultaneity”  clarification  increased 
available  power,  and  cooling  needs  of  the  system.  The  need  for  increased  power  and  cooling 
increased  the  size  and  weight  of  the  cooling  system  and  would  require  the  addition  of  a  larger, 
heavier  secondary  power  system  on  the  aircraft.  This  further  exacerbated  the  overweight  issue 
the  ACS  system  design  was  already  experiencing. 
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Pitfall/Lesson  Learned  -  Joint  Programs:  Lack  of  Well  Defined 
Requirements/Requirements  Changes  after  Funding  Lock 

While  Joint  Service  Acquisition  Programs  have  the  potential  to  limit  overall  cost  of 
capability  acquisition  for  DoD  due  to  economies  of  scale  caused  by  combining  resources, 
engineering  talent,  and  reducing  logistic  foot  print,  [451  they  also  have  the  potential  pitfall  of 
causing  programs  to  overrun  in  cost  and  schedule [461.  If  requirements  are  not  fully  vetted  by 
each  service  jointly  from  the  program  inception  the  opportunity  for  a  differing  interpretation  of 
requirements  is  likely.  Specifically,  “Requirement’s  thresholds  and  objectives  need  to  be  agreed 
to  by  each  service  early  to  lead  to  efficiency  and  effective”  program  execution[471.  However, 
“instability  in  requirements  is  the  hall  mark  of  joint  programs”[481.  By  the  time  the  Navy  joined 
the  ACS  program,  the  ORD  had  already  received  AROC  approval.  The  two  additional  operator 
workstations  to  meet  Navy  needs  were  added  prior  to  the  ORD  going  before  the  JROC  and  after 
funding  for  the  early  phase  of  system  development  was  locked.  Additionally,  the  Navy’s 
developing  CONOPS  required  that  the  SATCOM  package  on  the  aircraft  be  capable  of  very  high 
data  rates  not  originally  conceived  in  the  TD  phase  system  designs.  This  data  rate  requirement 
was  added  to  the  Performance  Based  Specification  provided  to  the  contractors  in  December  2003 
as  part  of  the  source  selection  package.  This  requirements  “clarification”  had  a  major  impact  on 
the  outer-mold-line  shape  of  the  aircraft  and  increased  its  drag  coefficient  that  reduced  the 
aircraft’s  endurance. 

Another  pitfall  that  must  be  considered  before  executing  a  joint  development  is  the 
impact  that  each  services  acquisition  philosophy  and  standard  operating  procedures  will  have  on 
program  execution.  As  discussed  previously,  these  operating  processes  are  not  the  same  among 
services  and  can  cause  friction  between  the  participating  services  during  a  development  effort. 
Additionally,  as  stated  by  the  Program  Executive  Officer  for  Intelligence  and  Electronic  Warfare 
Systems,  “Without  establishing  unity  of  command  through  a  single  joint  PM  office  with  defined 
service  roles  and  responsibilities  and  with  a  single  funding  stream,  service  parochialism  and 
competing  needs  will  lessen  service  commitment”  [49].  With  respect  to  the  Army  and  the  Navy 
on  ACS,  the  difference  in  acquisition  philosophy  and  lack  of  command  unity  led  to  an  impasse 
on  which  capability  trades  were  possible  as  the  Government  team  attempted  to  solve  the  payload 
weight  issue  while  attempting  to  remain  within  APB  cost  and  schedule. 
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Pitfall/Lesson  Learned  -  Design  Maturity:  Lack  of  Well  Defined  Requirements 

The  DoDI  5000.2  states  “The  project  shall  exit  Technology  Development  when  an 
affordable  increment  of  militarily-useful  capability  has  been  identified,  the  technology  for  that 
increment  has  been  demonstrated  in  a  relevant  environment,  and  a  system  can  be  developed  for 
production  within  a  short  timeframe  (normally  less  than  five  years).”[501  During  the  TD  phase, 
the  ACS  program  was  forced  into  a  situation  that  required  the  program  to  focus  its  technology 
risk  mitigation  dollars  on  maturing  critical  SIGINT  technology  that  was  initially  to  be  provided 
as  Government  Furnished  Equipment.  This  combined  with  the  budget  cut  that  the  program 
sustained  during  the  TD  phase  gave  rise  to  a  reduction  in  the  overall  breadth  of  ACS  system 
design  maturation  and  focused  this  phase  on  critical  technologies  development  and 
demonstration.  However,  based  on  the  definition  above,  it  can  be  argued  that  what  occurred 
during  the  ACS  TD  phase  is  in  keeping  with  the  expected  level  of  design  maturity  in  accordance 
with  5000  directive.  This  notion  is  further  reinforced  by  the  definition  of  what  is  expected  to 
occur  during  the  SDD  phase.  Specifically,  DoDI  5000.2  states  “The  purpose  of  the  SDD  phase  is 
to  develop  a  system  or  an  increment  of  capability;  reduce  integration  and  manufacturing  risk 
(technology  risk  reduction  occurs  during  Technology  Development);  . .  .”[5 11  The  SDD  phase 
then  is  the  point  at  which  the  system  design  is  matured,  and  the  system  developed  to  reduce 
integration  and  manufacturing  risk.  However,  in  the  ACS  program  it  was  precisely  the  lack  of  a 
system  design  before  the  start  of  SDD  that  left  unknown  many  of  the  SWAP-C  issues  that 
contributed  to  the  payload  being  overweight  for  the  aircraft.  It  was  the  detailed  system  design 
work  accomplished  immediately  after  the  initial  weight  issue  was  discovered  that  allowed  the 
Government/contractor  to  quickly  determine  the  extent  of  the  payload  weight  issue  and  avoid 
procuring  any  of  the  system’s  intended  aircraft.  While  system  design  in  the  TD  phase  might  have 
brought  to  light  the  payload  weight  issues  earlier,  it  would  have  required  additional  funding 
during  the  TD  phase  to  accomplish  that  level  of  effort.  However,  a  detailed  design  earlier  would 
have  provided  a  better  understanding  of  ACS  system  SWAP-C  requirements  and  allowed  for  a 
more  accurate  estimates  of  the  SDD  program  which  may  have  saved  schedule  and  ultimately 
dollars  in  the  end. 
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Pitfall  -  COTS/GOTS/NDI:  Lack  of  Well  Defined  Requirements/Requirements  Changes 
after  Funding  Locked 

The  concept  of  using  COTS/GOTS/NDI  was  meant  to  save  money  on  the  ACS 
development  program.  However,  due  to  unintended  consequences  of  these  items,  the  program 
struggled  to  close  on  a  system  design  that  would  provide  all  of  the  user  requested  capability.  As 
the  extent  of  the  SWAP-C  issue  became  clear,  what  also  became  clear  was  the  use  of 
COTS/GOTS/NDI  limited  the  contractor’s  ability  to  make  weight  savings  trades  while 
maintaining  full  capability  and  expected  cost.  The  use  of  COTS  aircraft  was  expected  to  save 
money  in  development  and  allow  for  less  costly  fleet  maintenance  during  operation  based  on 
parts  commonality  with  the  worldwide  fleet  in  the  airline  industry.  However,  the  requirement  for 
a  military  specification  based  Aircraft  Qualification  Plan  (AQP)  by  the  Government  had  a 
significant  impact  on  this  philosophy  and  the  ACS  system  design.  The  contractor  was  allowed  to 
modify  this  plan  during  the  RFP  process.  However,  when  returned  to  the  Government  the 
modified  plans  were  considered  inadequate  and  not  acceptable  to  the  Government.  Because  of 
lack  of  communications  between  the  contractor  and  the  Government  in  the  final  stages  of  the 
source  selection  process,  the  contractor  considered  that  the  Government  had  accepted  their 
version  of  the  AQP.  Based  on  this,  and  the  fact  that  full  AQP  requirements  implementation 
would  have  cost  considerably  more  than  had  been  budgeted  for  this  effort,  the  Government  and 
contractor  were  still  negotiating  the  details  of  the  ACS  AQP  at  contract  termination.  Also,  the 
extent  of  the  expected  cost  savings  for  using  COTS/GOTS/NDI  software  was  limited  because 
much  of  the  code  needed  to  be  modified  for  use  on  ACS.  The  use  of  COTS/GOTS/NDI  products 
is  not  a  negative  and  can  have  the  benefit  of  reducing  cost  and  saving  schedule.  However,  the  use 
of  these  products  without  first  having  a  detailed  system  design  can  lead  to  the  issues  faced  by  the 
ACS  program. 

ACS  also  encountered  other  pitfalls  during  the  execution  of  the  Acquisition  Process  that 
while  they  did  not  have  a  direct  cause  on  the  mismatch  between  funding  and  requirements,  they 
did  impact  the  efficient  execution  of  the  development  effort  and  hence  had  a  direct  impact  on 
program  cost  and  schedule.  These  pitfalls  cover  areas  that  must  be  understood  and  considered  as 
part  of  the  natural  progression  of  a  program  as  it  moves  from  one  phase  to  the  next  in  the 
Acquisition  Process.  By  doing  so,  one  realizes  that  the  complex  nature  of  the  process  includes 
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circumstances  over  which  the  PM  has  little  control  but  must  consider  if  he  is  to  develop  an 
comprehensive  acquisition  approach  and  ultimately  meet  cost,  schedule  and  performance. 

Pitfall  -  Gap  Between  Acquisition  Phases 

The  ACS  program  experienced  a  14-month  gap  between  the  end  of  the  TD  phase  and  the 
start  of  the  SDD  phase.  The  reason  for  the  gap  as  mentioned  earlier  was  that  the  Government 
team  was  accomplishing  items  and  events  necessary  to  execute  a  contract  award.  Specifically, 
the  team  accomplished  an  ASARC,  developed  documentation  to  meet  the  DoDI  5000.2  MS  B 
requirements/program  execution,  incorporated  the  Navy  into  the  Army  program  management 
team,  published  a  RFP,  executed  the  SDD  Source  Selection  Process,  developed  a  contract, 
accomplished  a  DAB  preparation  OIPT  and  ultimately  executed  a  “paper”  DAB.  While  this 
length  of  time  may  be  unique  to  ACS  because  of  the  system’s  complexity — six  months  was  spent 
by  the  program  office  educating  the  CAIG  analyst  on  what  ACS  was  and  the  extent  of  each 
subsystem’s  capability  so  that  he  felt  capable  of  providing  a  program  cost — there  is  a  break,  for 
some  length  of  time,  between  the  end  of  the  TD  phase  and  the  start  of  the  SDD  phase  for  all 
programs.  This  period  of  time  must  be  accounted  for  because  it  impacts  the  contractor’s  ability  to 
maintain  critical  team  members  and  hence  corporate  knowledge  of  the  program  and  their 
proposed  system  design.  During  the  ACS  TD  phase,  the  competing  contractors  requested  a 
“bridging  contract”  to  allow  work  to  continue  on  their  designs  during  the  interim  period  between 
phases.  At  the  time,  the  Government  did  not  have  the  funding  to  execute  this  request.  Because  of 
the  14-month  break  between  phases  of  the  ACS  program,  many  of  the  critical  team  members 
were  shifted  to  other  ongoing  programs  within  the  contractor’s  organization.  This  severely 
hampered  the  contractor’s  ability  to  ramp-up  development  execution  at  the  start  of  SDD  to  meet 
the  ACS  program  milestone  timelines. 

Pitfall  -  Increase  in  Program  Magnitude  between  Acquisition  Phases 

As  ACS  transitioned  from  the  TD  to  the  SDD  phase,  it  increased  from  an  ACAT  III 
program  to  an  ACAT  ID  program.  This  impacted  the  contractor  by  forcing  him  to  change  out 
most  key  leaders  on  the  program  after  contract  award  to  put  in  place  personnel  with  the 
experience  to  handle  the  complexities  inherent  in  an  ACAT  ID  program.  The  result  of  this  was 
the  loss  of  historical  knowledge  on  the  program  that  slowed  the  contractor’s  execution  of  critical 
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program  tasks  while  those  with  program  knowledge  trained  new  personnel.  Additionally,  the 
magnitude  of  the  increase  of  personnel  from  40  in  TD  to  400  in  the  first  six  months  of  SDD, 
created  chaos  that  added  to  an  already  complex  situation. 

This  type  of  personnel  switch  also  occurred  on  Government  team  but  to  a  lesser  extent. 
However,  this  change  did  create  a  ramp-up  period  for  the  Government  team  in  the  early  portion 
of  SDD  as  new  members  from  both  the  Army  and  Navy  were  added.  In  the  post  analysis  of  the 
program,  the  recommendation  was  made  that  the  Government  management  of  the  program 
should  have  transitioned  from  an  Intelligence  Battle  Operating  System  (BOS)  focused  PEO  to  an 
Aircraft  BOS  organization  to  ensure  a  better  understanding  of  the  complexities  involved  in 
aircraft  integration.  Specifically  noted  was  that  the  “program  structure  and  staffing  were  not 
consistent  with  technical  and  schedule  complexities  of  an  aircraft  ACAT  ID  program”[521.  This 
is  an  interesting  recommendation  considering  that  the  major  program  issue  was  the  weight  of  the 
payload  design  to  achieve  required  capability.  However,  the  development  and  addition  of  the 
payload  on  the  aircraft  brought  with  it  required  changes  to  the  outer-mold-line  of  the  aircraft  and 
in  several  proposed  designs  to  mitigate  the  weight  issue,  subtraction  of  fuselage  sections  and 
extensions  to  aircraft  wings.  What  this  demonstrates  in  hindsight  is  that  the  ACS  system  design 
was  not  as  mature  and  contained  much  more  risk  than  it  should  have  prior  to  the  commitment  of 
program  cost,  schedule  and  performance  more  so  than  which  organization  should  have  been  the 
program  management  lead[53]. 

Pitfall  -  Performance  Based  Contracting 

Based  on  DoDI  5000.2  guidance,  the  ACS  program  concept  relied  heavily  upon  the 
concept  of  performance  based  contracting.  Considering  this,  the  Government  issued  a  Statement 
of  Objectives  (SOO)  discussing  “what”  needed  to  be  accomplished  during  SDD  with  the 
expectation  that  the  contractor  would  develop  a  detailed  Statement  of  Work  (SOW)  describing 
“how”  they  would  accomplish  the  effort.  To  ensure  that  the  contractor  had  the  ability  to  create  an 
effective  SOW,  the  SOO  required  that  the  contractor  and  critical  subcontractors  be  certified  at 
least  at  a  Capability  Maturity  Model  Integrated  (CMMI)  level  3  at  the  start  of  SDD.  CMMI  level 
3  requires  that  an  organization  have  mature  processes  in  place  by  which  to  accomplish  work 
efforts.  While  the  contractor  claimed  to  be  at  this  level — the  contractor’s  corporate  headquarters 
had  recently  been  certified  at  the  higher  CMMI  level  5 — what  became  apparent  after  contract 
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award  was  that  the  contractor  team  with  responsibility  for  the  ACS  program  was  not  operating  at 
any  discernible  CMMI  level.  This  was  due  in  large  part  to  the  change  in  personnel  and  the 
training  level  of  the  new  team  on  this  corporate  division’s  processes.  Based  on  an  internal 
contractor  analysis,  they  expected  to  begin  operating  at  a  CMMI  level  2-3  within  a  year  and  a 
half  after  contract  award.  This  played  a  major  role  in  contractor’s  inability  to  execute  in  an 
efficient  and  competent  manner  as  seen  by  the  Government  adding  to  the  potential  for  expected 
cost  and  schedule  growth[541[55]. 

Pitfall  -  Competitive  Optimistic  Contractor  Proposals 

The  ACS  program  suffered  from  a  reoccurring  situation  within  the  current  Defense 
Acquisition  Process.  What  became  evident  after  contract  award  and  during  open  program 
execution  discussions  was  that  the  contractor’s  proposed  costs  were  overly  optimistic.  The 
contractor  made  it  clear  after  award  that  due  to  the  competitive  environment  of  the  source 
selection,  the  program  cost  bid  in  the  final  proposal  was  a  “win”  or  “buy-in”  cost  accurate  only  if 
the  program  executed  with  very  few  if  any  of  the  expected  program  risks  being  realized. 
Considering  that  historically  the  Government  normally  continues  large  programs  even  after  they 
have  breached  their  APB,  adding  additional  funding  and  then  reestablishing  the  program 
baseline,  it  appears  contractors  have  very  little  motivation  to  provide  realistic  expected  costs  in 
their  proposals.  Part  of  the  blame  for  this  situation  in  ACS  could  rest  with  the  Government 
because  they  supplied  in  the  RFP  the  available  budget  for  each  year  of  expected  SDD  contract 
execution.  However,  this  was  done  in  an  attempt  to  ensure  the  competing  contractors  provided  a 
best  value  proposal  considering  available  cost,  schedule  and  requested  capability.  In  the  recently 
published  DAPA  study,  a  new  approach  is  discussed  that  would  fix  the  parameters  of  estimated 
program  cost  and  schedule,  and  only  allow  the  changing  of  program  performance  for  a  given 
development  spiral  to  ensure  that  the  first  two  parameters  are  achieved[561.  This  could  fix  the 
issue  of  cost  breaches  in  the  future. 

Pitfall  -  Pre  MS  B  Program  Readiness  Assessment 

In  the  execution  of  the  current  Acquisition  Process,  the  ACS  program  accomplished  an 
AS  ARC,  several  OSD  IIPTs,  an  OSD  OIPT  and  a  “paper”  DAB.  What  becomes  apparent 
looking  back  on  these  meetings  and  the  additional  reviews  prior  to  these  events  is  that  the 
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program  was  never  reviewed  in  a  detailed  synergistic  fashion  until  the  program  was  experiencing 
development  issues.  Prior  to  MS  B,  each  subject  matter  expert  (SME)  that  reviewed  the  various 
program  aspects/documents  did  so  without  detailed  consideration  for  the  other  areas  of  the 
program.  The  leadership  at  the  major  reviews  mentioned  above  were  presented  with  a 
comprehensive  briefing  of  all  aspects  of  the  program,  but  had  to  rely  on  the  SMEs  to  accomplish 
the  detailed  analysis  that  provided  insight  into  possible  problem  areas.  With  each  SME  returning 
a  vote  of  confidence  in  his  or  her  area  of  responsibility,  this  naturally  provided  the  leadership  a 
positive  assurance  that  the  program  was  prepared  for  entry  into  SDD.  Considering  the  results  of 
the  Independent  NAR  Team’s  assessment  of  the  ACS  program,  it  is  apparent  that  had  a 
synergistic  review  approach  been  taken  prior  to  MS  B,  several  of  the  major  program  disconnects 
would  have  been  discovered.  Key  to  the  success  of  this  approach  is  the  experience  level  of  the 
independent  review  team  members  and  their  focus  areas.  In  the  case  of  ACS,  the  assembled 
NAR  team  was  composed  of  very  senior  acquisition  professionals  who  had  “a  combined  700 
years  of  experience”  from  both  the  Army  and  the  Navy.  Their  focus  was  to: 

-  Assess  program  scope  with  respect  to  cost/schedule/performance  (including 
Program  Management/Organization/Experience,  risks,  acquisition  strategy,  etc) 

-Provide  “pre-decisional”  findings  and  recommendations  to  Army  and  Navy  PEOs. 

-  Suggest  “path  forward” [571. 

Pitfall  -  Corporate  Sponsorship  for  Acquisition  Programs 

While  important,  the  level  of  Corporate  Sponsorship  in  the  civilian  industry  sector  is  not 
equaled  within  DoD.  Civilian  sector  corporate  leadership  marshals  all  resources  requested  by  a 
program  manager  to  develop  and  bring  a  product  to  market  as  fast  as  possible.  Therefore,  the 
only  focus  area  of  a  civilian  PM  is  bringing  the  product  to  market.  In  DoD,  there  is  not  the  same 
alignment  of  corporate  sponsorship  for  product  development.  In  DoD,  corporate  sponsorship  is 
accomplished  by  what  is  called  the  big  “A”  (“A”  being  the  Acquisition  Community  which 
includes  not  only  the  PM/Acquisition  work  force,  but  also  the  senior  Army,  OSD  leadership  and 
Congress  that  have  purview  over  the  resources  for  product  development).  However,  currently  in 
DoD  the  differing  focuses  of  the  big  “A”  members  does  not  usually  align  sponsorship  for 
development  projects  on  the  single  goal  of  bringing  a  product  to  the  customer.  Without  this  big 
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“A”  sponsorship,  the  DoD  PM  must  focus  on  developing  and  bringing  to  market  his  product  and 
also  attaining  and  then  defending  the  necessary  resources  to  make  this  happen.  Because  of  this 
lack  of  big  “A”  sponsorship  the  DAPA  report  stated,  “no  program  has  ever  been  plused  up  in 
funding  during  development”[581.  To  this  point,  the  opposite  appears  to  be  occurring;  the 
Assistant  Secretary  of  the  Army  for  Acquisition  Logistics  and  Technology  recently  said,  “the 
Future  Combat  System  has  been  cut  $830M  since  its  MS  B  decision  in  2003  and  it  is  a  successful 
program  having  remained  on  cost  and  schedule  since  its  inception” [591.  To  minimize  the  impact 
of  the  current  lack  of  big  “A”  sponsorship,  a  program  must  get  to  production  as  soon  as  possible. 
To  accomplish  this,  technology  for  a  single  development  spiral  must  be  at  a  level  of  maturity  to 
reduce  time  for  product  development. 

Why  Pitfalls  Happen 

Considering  that  DoD  is  executing  a  knowledge-based  acquisition  process  that  mirrors 
the  successful  commercial  sector’s  process,  why  is  it  that  the  ACS  program  suffered  from  the 
pitfalls  that  it  did?  And  more  generally,  why  is  it  that  of  the  85  ACAT  ID  programs  reported  in 
the  FY06  Selected  Acquisition  Report,  40  of  them  reported  Nunn-McCurdy  unit  cost  breaches 
with  25  of  those  reporting  a  breach  of  50%  or  more  compared  to  their  original  cost  baselines? 
Congress  considers  the  situation  so  bad  that  in  the  first  sentence  of  the  HASC  FY2007  National 
Defense  Authorization  Act,  Acquisition  Policy  and  Management  section  they  stated  emphatically 
“...the  Department  of  Defense  (DOD)  acquisition  process  is  broken.”  The  second  sentence  is 
even  more  damning  stating  “The  ability  of  the  Department  to  conduct  the  large  scale  acquisitions 
required  to  ensure  our  future  national  security  is  a  concern  of  the  committee.”  They  further  cite 
that  “the  rising  costs  and  lengthening  schedules  of  major  defense  acquisition  programs  lead  to 
more  expensive  platforms  fielded  in  fewer  numbers.”  The  very  issues  that  the  current  Defense 
Acquisition  System  was  suppose  to  fix  are  still  issues.  The  question  is  why? 

Answers  to  the  above  questions  might  be  found  in  February  2006  DAPA  report 
commissioned  by  Deputy  Secretary  of  Defense.  As  discussed  in  the  previous  section  but  with 
further  elaboration  here,  the  DAPA  team  of  experienced  acquisition  professionals  recognized 
that  the  Defense  Acquisition  System  is  a  complex  integration  of  three  competing  processes. 

These  three  competing  processes  are  the  acquisition,  budgeting  and  requirements  processes 
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which  they  term  collectively  as  the  “big  A”  acquisition  system.  The  current  DoDD  5000.1  based 
acquisition  process  they  term  as  the  “little  a”  stating  that  it  does  not  include  the  requirements  and 
budget  processes.  Figure  3  provides  a  graphical  depiction  of  this[60] . 
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Figure  3.  DAPA’s  Acquisition  System 
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This  competition  within  the  “big  A”  system  is  caused  by  the  differing  component 
process’  motivational  factors  described  in  the  red  and  black  text  in  Figure  4.  While  in  theory 
there  exists  a  workforce,  organizations  and  industry  which  should  work  together  to  force  the 
three  separate  processes  of  “big  A”  into  synergy  and  ultimately  develop  a  needed  warfighter 
capability  on  target  cost  and  with  defined  schedule,  in  actuality  these  “forcing  entities”  have 
competing  values  of  their  own  which  make  this  impossible.  Figure  4  shows  this  concept 
providing  the  differing  values  in  blue[61]. 
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ORGANIZATIONAL  VALUES  DIFFER 


I  here  are  fundamental  disconnects,  in  LtoL?  ma^age*nent  syitema  and  Congressional  QYefs-lght  driven  by 
eonoeting  values  and  objectives  five  i  create  gouemmenunditceo  instability  n  orr acquisition  programs 


Klrt’MiiCi-IAMlIfthFh  TO  Pill  Y 

CQsTRO. .  OvFBSJGhT  AhP  RAl  AhCS  IHSTAfi  I  (TV  ThAT  C^FaTRS 


,  aTA|MLrrf 
■  TU'.V  hC, 

-  JESSAFSRAZTDri 

-  HFEREMOE 


r:a*nnc'' 


WMt  AMD  ftl-AT  TD  E-.r-- 
■  KISilWI  EUCCtSE  AT 


-«YV  :  3  BLV 

.  EA.AICE  OF  CCST 


LCvrt.-5'r  ;kt  k  lfe 


EtKEOULE  AhDFLRFCRVtAHCE 


SFflvVr  AOWCACt 


TFCaiSOLWt  An-Ail  ABl.F™ 
PFFGFWJV^F  CGKTAJiD 
T  sir  TO  MFFri 


i  SURVIVAL  ■  FCKKHCIUDEH  ittLilE 

<  FMECIITABLirY  ..  iTHDtSi  WTAUtV 


In  Practice,  Disconnected  and  Unstable  (government  induced) 


Figure  4.  DAPA’s  Organizational  Values  Differ.. 


These  competing  motivational  factors  of  the  “big  A”  processes  and  competing  values  of 
the  forcing  functions  cause  instability  in  the  Defense  Acquisition  System  that  they  define  as 
being  comprised  of  the  six  internal  elements  of  budget,  requirements,  acquisition,  industry 
workforce  and  organization.  This  instability  in  the  Acquisition  System  is  what  they  claim 
“results  in  a  situation  in  which  senior  leaders  in  the  Department  of  Defense  and  Congress  are 
unable  to  anticipate  or  predict  the  outcome  of  programs  as  measured  by  cost,  schedule  and 
performance”[62].  Eight  Major  Findings  resulted  when  the  DAPA  “reviewed  the  defense 
acquisition  performance  and  documented  the  integrated  nature  of  the  process.”  These  are  listed 
in  Figure  5[63]. 
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MAJOR  FINDINGS 
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Figure5.  DAPA’s  Major  Findings 


Based  on  these  major  findings  the  DAPA  project  made  reform  recommendations 
designed  to  improve  all  of  the  six  internal  elements  of  the  Acquisition  System.  The  summary  of 
each  of  the  recommendations  is  found  in  Figure  6  [64]. 
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•  Organization 

-  Reaign  authority,  accountably  and  responsibility  at  the  appropriate  level  and  streamline  the  acquisition 
oversight  process 

•  Establish  defeated  Four-Star  Acquisition  Systems  Commands,  at  the  Service  level 

•  Workforce 

-  Rebuild  and  value  the  acqutsiton  workforce,  and  inoentr.ized  leadershp. 

•  Budget 

-  Transform  the  flaming,  Programming  and  Budgeting  process  and  eslabish  a  distinct 
Stable  Program  Fundng  Account 

•  Requirements 

-  Replace  the  Joint  Capability  Integrator  and  Development  System  with  the  Joint  Capacities  Acquisition 
and  Divestment  Plan  (a  Combatant  Commander-led  requirements  process  in  which  the  Services  and 
Defense  Agencies  compete  to  provide  solutions.) 

•  Establish  a  two-year  recurring  process  to  produce  an  integrated,  time-phased  and  fiscally-informed 
Joint  Capacity  Acqusition  and  Divestment  plan  and  a  continuous  Matenel  Solutons  Plan  Development 
Process  to  identify  and  initiate  development  of  Materel  Solutions 

•  Add  an ' Opera tionaly  Acceptable’  test  evaluator  category. 

-  Gve  program  managers  explicit  autiority  to  defer  non-Key  Performance  Parameter  requirements  to 
later  sprats  or  block  upgrades. 

•  Acquisition 

•  Adopt  a  risk-based  source  selection  process 

-  Shift  to  tme-oertain  development  procedures  and  make  schedule  a  Key  Performance  Parameter. 

•  Mandate  a  tme  start  and  end  dates  that  are  dearly  defned  and  revamp  the  acquisition 
processes  to  support  it 

•  Industry 

•  Overcome  the  consequences  of  reduced  demand  by  sharing  long  range  plans  and  restructuring 
competitors  tor  new  programs 

•  Require  government  insight  and  favor  formal  oompetition  for  major  subsystems  when  a  Lead  System 
Integrator  acquisition  strategy  is  pursued 

Figure  6.  Overview  of  DAPA’s  Findings. 


Their  review  of  acquisition  reform  history  shows  that  many  of  these  issue  areas  contained 


in  their  major  findings  had  been  incrementally  addressed  over  time  via  similar  recommendations 
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in  previous  calls  for  acquisition  reform.  However,  DAPA  notes  that  for  the  incremental  approach 
of  acquisition  reform  to  be  successful  in  improving  cost,  schedule  and  performance  it  requires 
that  “all  six  internal  elements  of  the  (Defense)  Acquisition  System  (organization,  workforce, 
budget,  requirements,  acquisition  and  industry)  must  operate  in  a  stable  and  predictable  manner.” 
Also,  they  stated  “that  external  influences  on  the  (Defense)  Acquisition  System,  including 
leadership  and  congressional  oversight,  must  exert  stabilizing  and  predictable  guidance.  None  of 
these  processes  and  influences  are  stable  and  predictable  today”[65].  Therefore,  they  conclude 
that  to  provide  the  Acquisition  System  the  required  “stability  and  continuity”  that  it  needs  to 
succeed,  the  approach  must  be  to  improve  all  six  internal  elements  (organization,  workforce, 
budget,  requirements,  acquisition  and  industry)  at  once.  Hence,  they  leave  one  to  conclude  that 
the  lack  of  simultaneous  implementation  of  previous  calls  for  reform  is  the  reason  for  their  lack 
of  success. 

Noted  in  the  DAPA  study  but  not  called  out  as  a  reason  for  the  failure  of  previous  reform 
studies  to  produce  a  successful  acquisition  process  is  that  not  all  elements  of  the  previous  studies 
were  fully  implemented.  Considering  that  implementing  improvements  in  all  the  six  elements 
simultaneously  is  the  only  way  the  DAPA  study  team  says  DoD  can  gain  the  needed  stability  in 
the  Acquisition  System  for  it  to  be  successful  in  predicting  program  cost,  schedule  and 
performance,  it  is  interesting  that  no  rationale  is  given  for  why  the  previous  reform  initiatives 
were  not  fully  implemented.  Specifically,  the  DAPA  study  noted  that  the  1986  President’s  Blue 
Ribbon  Commission  on  Defense  Management  National  Security  Planning  and  Budgeting 
(Packard  Commission)  call  for  organizational  reform — while  similar  to  their  own — was  never 
fully  implemented.  DAPA  stated  that  the  Packard  Commission  recognized  that  successful 
organizations  have  “short,  unambiguous  lines  of  communication  among  levels  of  management, 
small  staffs  of  highly  competent  professional  personnel .  .  .  [and]  most  importantly,  a  stable 
environment  of  planning  and  funding.”  The  DAPA  study  found  that  this  concept  was  never  fully 
implemented  and  recommended  as  its  first  measure  for  organizational  reform  “. .  .to  implement 
the  intent  of  the  Packard  Commission  more  fully  and  regain  stability  in  the  Acquisition  System 
by  realigning  authority,  accountability  and  responsibility  at  the  appropriate  levels”[661. 

Reviewing  the  Packard  Commission’s  study  reveals  that  it  also  found  that  the  previous 
calls  for  reform  were  not  fully  implemented.  Specifically  the  commission  noted  that  the  reform 
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initiatives  called  for  by  President  Eisenhower  during  his  presidency  based  on  lessons  learned 
from  World  War  II  were  never  fully  instituted.  The  Commission  stated: 


“The  present  structure  of  the  Department  of  Defense  (DoD)  was  established  by  President 
Eisenhower  in  1958.  His  proposed  reforms,  which  sprang  from  the  hard  lessons  of 
command  in  World  War  I  and  from  the  rich  experience  of  his  Presidency,  were  not  fully 
accomplished.  Intervening  years  have  confirmed  the  soundness  of  President 
Eisenhower’s  purposes.  The  Commission  has  sought  to  advance  on  the  objectives  he  set 
for  DoD” [67], 

Like  the  DAPA  study  the  Packard  Commission  also  claimed  as  a  goal  the  desire  to 
“advance  on  the  objectives”  that  their  predecessor  had  set  with  respect  to  acquisition  reform. 
Immediately  following  the  publishing  of  both  the  Packard  Commission’s  interim  and  final 
reports  this  seemed  to  be  occurring.  After  the  reports  were  published  there  was  a  flurry  of 
activity  by  both  the  Congress  and  DoD  to  implement  several  of  the  reform  initiatives.  However, 
in  the  end,  the  reform  initiatives  were  only  ever  partially  implemented[681.  The  same  set  of 
events  appear  to  be  occurring  with  the  DAPA  report.  In  the  2007  National  Defense  Authorization 
Act,  the  House  Armed  Services  Committee  (HASC)  commented  on  many  of  the  major  findings 
of  the  DAPA  study  recommending  that  DoD  should  be  implement  the  findings.  Section  804  of 
the  Act  goes  further  and  requires  that  DoD  provide  quarterly  reports  to  the  HASC  and  Senate 
Armed  Services  Committee  on  the  DoD’s  implementation  of  the  DAPA,  The  Defense  Science 
Board  Summer  Study  on  Transformation,  The  Center  for  Strategic  and  International  Studies: 
Beyond  Goldwater-Nichols  Study,  The  Quadrennial  Defense  Review  and  The  Committee 
Defense  Review  of  the  House  Committee  on  Armed  Services  recommendations.  However,  in  the 
next  paragraph  of  the  section  the  committee  calls  into  question  the  Defense  Departments  ability 
to  “analyze  and  synthesize  these  reform  recommendations  into  a  series  of  meaningful  and 
actionable  implementation  plans.”  They  state  that  in  the  last  20  years  DoD  has  been  unable  act 
on  the  recommendations  made  by  the  numerous  reform  studies  and  that  the  “same  challenges 
identified  by  the  Packard  Commission  including  rampant  cost  growth,  unreliable  cost  estimates, 
and  requirements  relying  on  immature  technology  increasing  overall  program  cost”  still  remain 
issues  today. 

“Bureaucratic  impediments,  changing  senior  leadership,  and  numerous  other  factors 
prevented  implementation  of  major  acquisition  reform  despite  comprehensive  studies  on 
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the  subject.  In  particular,  the  committee  notes  that  the  President’s  Blue  Ribbon 
Commission  on  Defense  (1986),  commonly  known  as  the  “Packard  Commission,” 
recommended  numerous  reforms  to  the  acquisition  system  that,  despite  the  efforts  of 
Congress  and  the  Department,  have  not  been  fully  realized.  Nearly  twenty  years  later,  the 
four  major  acquisition  reform  studies  of  2005  identify  the  same  challenges  identified  by 
the  “Packard  Commission”  including  rampant  cost  growth,  unreliable  cost  estimates, 
and  requirements  relying  on  immature  technology  increasing  overall  program  cost.  The 
committee  is  concerned  about  the  ability  of  the  Department  to  solve  these  decades’  old 
problems” (69].  — HASC  2007  National  Defense  Authorization  Act 

While  calling  into  question  the  DoD’s  ability  to  implement  acquisition  reform,  the  Act’s 
language  does  little  to  impose  reform  changes  itself.  For  example,  while  Section  813  of  the  2007 
Act  language  does  require  that  DoD  implement  “time-certain  development”  as  called  out  by  the 
DAPA  study;  it  does  so  only  for  Information  Technology  Business  Systems.  The  language 
requires  that  these  “...systems  be  fielded  within  five  years  of  the  system  entering  the  technology 
development  phase  of  the  acquisition  process,  known  as  Milestone  A  approval”[70].  Discussion 
of  one  of  the  most  far-reaching  DAPA  reform  initiative  is  nowhere  to  be  found.  The  major 
organization  change  initiative  called  for  in  the  DAPA  study  of  establishing  four-star  System 
Commands  for  Acquisition  under  each  service’s  Chief  of  Staff/Chief  of  Naval  Operations  to 
unify  acquisition,  requirement,  and  budget  process  command  authority  is  not  even  mentioned  by 
the  Act. 

So  while  Congress  is  requesting  action  by  DoD  to  fully  implement  the  reform  initiative 
called  for  by  the  DAPA  report  and  others,  it  has  little  faith  this  will  happen.  And,  knowing  this 
Congress  has  taken  few  steps  to  implement  these  initiatives.  Based  on  the  historical  record,  the 
highest  probability  of  outcome  is  that  only  a  portion  of  DAPA’s  initiatives  will  be  implemented 
within  DoD  as  has  been  the  case  with  all  other  reform  initiatives  in  the  past.  Therefore,  while  the 
DAPA  study  may  explain  why  the  current  Acquisition  Process  is  unable  to  reliably  execute 
major  defense  acquisition  programs  within  cost,  and  schedule  for  the  required  performance,  it 
will  probably  do  little  to  change  this  situation. 

Experience  Counts 

The  reality  is  that  the  Acquisition  Process  is  not  as  “broken”  as  has  been  stated  above.  If 
the  process  were  truly  broken  as  described,  we  would  expect  that  DoD  would  never  be  able  to 
execute  an  AC  AT  ID  program  and  provide  required  capability  within  budget  and  on  schedule. 
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This  is  not  the  case.  There  are  in  fact  large  acquisition  programs  that  are  executing  within  the 

Acquisition  Program  Baseline  (APB)  parameters  set  at  the  start  of  the  program’s  SDD.  The  same 

SAR  data  previously  discussed  shows  this  fact.  While  the  final  SAR  submission  of  FY06  showed 

that  of  the  85  programs  submitted  40  of  those  were  in  Nunn-McCurdy  breach  of  their  APB,  what 

it  also  says  is  that  there  were  45  programs  which  were  not  in  breach  of  their  APBs.  Considering 

this  and  the  fact  that  two  of  the  largest  ACAT  ID  acquisition  programs  in  both  the  Army  and  the 

Navy  (Future  Combat  System  and  Multimode  Maritime  Aircraft)  are  executing  within 

established  parameters, [71]  several  reviewers  of  the  Defense  Acquisition  System  have  said  that 

it’s  not  the  Defense  Acquisition  Process  at  all  but  instead  the  “experience  and  discipline”  of  the 

PMs  and  the  acquisition  leadership  to  effectively  implement  the  Acquisition  Process  that  is 

causing  the  current  situation[72][731.  As  a  matter  of  fact,  comments  concerning  PM/acquisition 

workforce  education  and  experience  levels  have  been  made  by  almost  every  major  acquisition 

system  reform  study  to  date.  In  the  DAPA  study — as  with  the  Packard  Commission — the 

researchers  noted  that  many  PMs  lack  the  “experience  and  expertise”  in  all  facets  of  program 

management  needed  to  successfully  execute  a  program  within  established  cost,  schedule  and 

performance.  Specifically,  the  two  studies  stated: 

“Experience  and  expertise  in  all  functional  areas  has  been  de-valued  and  contributes  to  a 
“Conspiracy  of  Hope”  in  which  we  understate  cost,  risk  and  technical  readiness  and,  as  a 
result,  embark  on  programs  that  are  not  executable  within  initial  estimates.  This  lack  of 
experience  and  expertise  is  especially  true  for  our  program  management  cadre”[74]. 

—DAPA 

“Each  year  billions  of  dollars  are  spent  more  or  less  efficiently,  based  on  the  competence 
and  experience  of  these  personnel.  Yet,  compared  to  its  industry  counterparts,  this 
workforce  is  under-trained,  underpaid,  and  inexperienced.  Whatever  other  changes  may 
be  made,  it  is  vitally  important  to  enhance  the  quality  of  the  defense  acquisition 
workforce” [751.  — Packard  Commission 

The  DAPA  report  also  notes  that  DoD  has  “no  consistent  training  or  experience  requirements. . . 
for  these  key  skills  and  training  and  certification  standards  are  not  enforced”[761. 

Based  on  these  finding,  it  is  evident  that  experience  levels  can  vary  among  PMs  and  the 
acquisition  leadership.  Based  on  this,  it  should  be  expected  that  PMs  might  not  have  the  requisite 
experience  before  entering  an  acquisition  program  in  a  particular  stage  of  the  acquisition  process. 
PMs  like  other  members  of  the  acquisition  and  military  workforce  are  placed  into  positions  of 
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ever  increasing  responsibility  based  on  their  past  performance  and  their  potential  future  ability. 
Although  they  may  have  superior  performance  and  potential,  without  the  appropriate  experience 
they  may  not  understand  the  potential  pitfalls  inherent  in  the  current  acquisition  process  at  a 
particular  phase.  The  Aerial  Common  Sensor  (ACS)  program  is  a  case  in  point.  Had  the  ACS 
PM  and  the  Army  leadership  been  aware  of  the  best  practice  process  pitfalls  of  the  current 
Defense  Acquisition  System  as  the  program  transitioned  from  TD  to  SDD  they  could  have 
avoided  some  if  not  all  of  the  issues  the  program  faced. 

Recommendations:  Consider  Experience  and  Process  Improvement 

An  obvious  way  to  correct  the  lack  of  experience  situation  in  the  future  might  be  to 
ensure  that  only  PMs  with  the  appropriate  experience  are  placed  in  complex  Major  Defense 
Acquisition  Programs  (MDAP)  in  a  particular  phase  of  the  acquisition  process.  While  this  is 
possible  from  a  centralized  career  management  stand  point,  it  is  probably  not  practical.  After  a 
few  rounds  of  matching  experienced  PMs  to  programs  in  particular  phases  of  the  acquisition 
process,  DoD  would  run  out  of  experienced  PM  because  of  the  “catch-twenty-two”  of  other  PMs 
not  being  able  to  gain  requisite  experiences  to  qualify  them  for  MDAP  programs.  While  the 
axiom  in  life  is  that  we  learn  more  from  our  mistakes  than  our  successes,  in  the  world  of  DoD 
MDAP  programs,  management  mistakes  can  be  costly.  Hence,  using  unqualified  PMs  can  keep 
DoD  in  the  current  situation. 

DoD  could  look  outside  the  Department  for  a  fresh  infusion  of  experienced  personnel. 
This  would  probably  not  be  optimal  for  two  reasons.  First  this  solution  would  not  provide  DoD 
PMs  a  worthwhile  career  progression  path.  Second,  PMs  from  outside  DoD  would  not  have  DoD 
specific  program  management  experience.  PM  experiences  are  just  that — his  or  her 
experiences — limited  to  the  experiences  that  they  have  had.  These  experiences  by  definition  are 
not  benefited  by  the  breadth  of  experiences  that  other  individuals  and  organizations  have  had. 
Also,  while  experience  is  a  “great  teacher”  the  level  of  lessons  learned  by  a  PM  from  his 
experiences  depends  on  “whether  the  student  was  paying  attention”[77]. 

A  solution  to  this  dilemma  is  education.  Specifically,  education  gained  by  combining 
training  on  current  Defense  Acquisition  System  theory  with  the  lessons  learned  from  PM 
experiences  with  the  pitfalls  of  the  Defense  Acquisition  System.  PMs  educated  in  this  way  would 
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understand  the  current  acquisition  process  to  include  the  benefits  of  the  required  best  practices 
and  lifecycle  documentation  that  have  proven  successful  in  commercial  industry.  And,  they 
would  also  understand  that  the  current  Acquisition  Process  has  pitfalls  that  must  be  mitigated  to 
succeed  in  developing  a  system  within  defined  cost,  schedule  and  performance.  By  doing  this, 
DoD  can  develop  knowledgeable  PMs  from  within  the  current  and  future  DoD  PM  pool  that  are 
prepared  to  optimize  a  program  for  success  even  without  personal  experience  with  all  the 
acquisition  process  pitfalls  themselves[78]. 

Considering  that  the  Defense  Acquisition  University  (DAU)  currently  trains  PMs  on  the 
Defense  Acquisition  System  theory,  it  would  only  need  to  add  sections  on  lessons  learned  by 
PMs  from  experiences  that  they  have  had  with  the  pitfalls  with  the  current  acquisition  process. 
These  experiences  can  be  captured  using  the  Army’s  After  Action  Review  (AAR)  process.  This 
process  has  been  a  very  integral  part  of  the  National  Training  Center’s  success  in  training  the 
world  class  Army  currently  fighting  in  Iraq.  This  process  has  also  been  used  in  commercial 
business  sector  as  part  of  their  process  for  reviewing  commercial  development  programs.  Bart 
Perkins,  who  is  a  managing  partner  at  Leverage  Partners  Inc.,  recently  published  in  his  monthly 
column  for  the  management  section  of  Computerworld  magazine  a  business  focused  synopsis  of 
the  AAR  process.  In  his  article,  the  process  is  called  Post  Project  reviews  (PPR).  Considering  its 
project  management  focus,  this  process  is  well  suited  for  use  in  the  business  of  Defense 
Acquisition.  Perkins  starts  by  stating  that  the  “cornerstones  of  a  successful  PPR  are  effective 
leadership,  thorough  preparation,  committed  participation  and  follow-through.”  For  effective 
leadership,  he  recognizes  that  “senior  project  managers  with  experience  and  perspective”  and 
who  are  free  of  the  program’s  “politics”  must  be  chosen  to  lead  the  effort.  This  can  be 
accomplished  in  DoD  by  assigning  a  team  of  senior  experienced  acquisition  personnel  from 
DAU  augmented  as  necessary  by  other  PMs  and  senior  acquisition  functional  area  specialist  to  a 
program  review  team. 

While  effective  leadership  is  important,  Perkins  states  that  thorough  preparation  must  not 
be  “minimized  or  eliminated”  as  it  often  is  because  it  is  a  “crucial  step”  in  the  success  of  a  PPR. 
Through  proper  preparation  the  effectiveness  of  the  PPR  is  “significantly  enhanced”.  It  also 
“shortens  the  length  of  PPR  meetings  and  reduces  the  number  of  arguments.”  To  prepare 
properly,  Perkins  says,  the  leadership  team  must: 
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-  Reassess  Project  Content:  review  program  “documents,  including  the  business  case, 
work  plans,  change  requests  and  contracts.  Critically  reexamine  business  goals,  project 
objectives  and  assumptions,  as  well  as  risks  and  associated  risk  mitigation  strategies.” 

-  Analyze  Statistics:  “Compare  initial  estimates  of  projects  costs,  schedule  and 
deliverables  with  actual  data.  Significant  differences  (increases  or  decreases)  usually  indicate 
insufficient  research  or  unforeseen  obstacles.  Gauge  overall  project  stability  by  assessing 
revisions  to  scope,  budget,  schedule  and  resources.” 

-  Interview  Project  Participants:  “Interview  key  project  team  members  and  executives 
to  add  important  perspective,  especially  for  contentious  issues.  Interviews  frequently  reveal 
unrealistic  (often  mandated)  deadlines,  overly  constrained  budgets,  insufficient  staffing, 
inadequately  tested  software  packages  or  corporate  in-fighting,  which  greatly  affect  project 
success.” 

-  Establish  PPR  Meeting  Logistics:  “Determine  participants  (in  PPR  meeting).  Include 
key  representatives  for  all  groups  that  worked  on  (or  benefited  from)  the  project,  as  well  as 
multiple  levels  of  management. . .”  “Multiple  meetings  or  videoconferences  may  be  required.  E- 
mail  is  not  acceptable.” 

Perkins  further  states  under  the  heading  of  Participation  that  “the  PPR  meeting  should 
address  these  questions: 

-  What  went  well,  and  what  didn’t? 

-  What  obstacles  impeded  the  project  and  what  would  have  helped? 

-  Where  do  processes  need  to  be  altered,  improved  or  replaced? 

-  How  effective  were  project  communication  and  executive  support?” 

To  succeed  these  meetings  must  “maintain  a  constructive,  non-adversarial  focus,  and  aim 
for  a  reasonable  level  of  consensus.” 

However  probably  the  most  important  part  to  the  PPR  process  is  the  follow  through.  Just 
having  information  about  a  program’s  highs  and  lows  does  not  provide  change.  “Results  must  be 
integrated  and  published. .  .without  conscious  effort  and  commitment  to  improvement,  few 
changes  will  occur  and  subsequent  PPRs  will  reveal  essentially  the  same  problems. .  .Many 
companies  assume  that  unsuccessful  projects  are  exempt,  but  failures  offer  invaluable 
opportunities  for  learning. .  .”[79]. 
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In  DoD,  PPRs  should  be  accomplished  when  a  MDAP  program  completes  a  phase  of  the 
acquisition  process,  at  program/contract  cancellation  and  following  a  program’s  execution  of  a 
Milestone  decision.  Conducting  a  PPR  at  the  end  of  a  phase  of  the  acquisition  life-cycle  and  at 
program/contract  cancellation  will  provide  specific  insight  into  the  peaks  and  pitfalls  that  a 
program  faced  while  those  events  are  still  fresh  in  the  minds  of  the  program  personnel  and 
relevant  program  documentation  is  readily  available.  Executing  a  PPR  following  a  Milestone 
decision  provides  feed  back  on  the  unwritten  approval  and  briefing  process  along  with  the 
memorandum  and  documentation  content  that  an  ACAT  ID  program  must  accomplish  in 
preparation  for  a  specific  Milestone  decision.  Creating  a  historical  record  that  others  can  use 
with  which  to  prepare  for  a  Milestone  decision  can  add  consistency  and  reduce  the  difficulty  a 
PM  currently  has  with  obtaining  useful  information  on  the  Milestone  briefing  process  in  a  timely 
manner.  As  a  PM  marches  down  the  path  to  a  Milestone  decision,  he  must  query  each 
organization  responsible  for  the  briefings  and  required  documentation  content  on  exactly  what  is 
acceptable.  Without  a  standard  operating  procedure  for  the  specifics  required  in  each 
organization’s  documents/briefing  charts,  the  PM  must  provide  content  that  is  to  the  satisfaction 
of  the  individual  action  officer  on  duty  at  the  time  of  his  Milestone  decision[801[81]. 

As  these  PPRs  are  accomplished,  the  information  obtained  can  be  used  in  developing  the 
DAU  course  curriculum  for  PMs  on  pitfalls  and  successes  of  ACAT  ID  or  MDAP  programs. 
Perhaps  this  curriculum  can  be  tailored  for  each  PM  based  on  the  ACAT  ID  program  and  the 
specific  phase  of  the  acquisition  process  the  program  is  in  or  will  be  entering.  This  curriculum 
should  be  added  to  the  current  courses  required  for  PMs  who  have  been  slated  to  manage  ACAT 
ID  programs.  Later  these  courses  could  be  expanded  to  include  ACAT  III  program  managers. 
However,  a  very  important  aspect  of  the  DoD  PPR  process  is  that  the  data  collected  from  this 
process  must  not  focus  on  placing  blame  or  have  as  an  end  goal  a  focus  of  providing  input  to  a 
PM’s  annual  performance  evaluation.  Should  this  happen,  the  process  will  no  longer  provide 
honest  feed  back  as  PMs  are  forced  to  influence  the  characterization  of  their  programs  to  avoid  a 
negative  performance  assessment. 

In  addition  to  creating  courses  that  include  PPR  information,  DAU  should  create  an 
online  Program  Expert  System,  available  to  all  PMs  that  database  the  collected  PPR  information. 
This  expert  system  would  make  available  the  relevant  PPR  information  in  a  format  that  a  sitting 
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PM  would  be  able  to  query.  Specifically,  the  system  would  provide  PMs  the  ability  to  parse  the 
information  based  on  several  defining  criteria  such  as  program  phase  and  system  specifics.  For 
example,  once  established,  a  PM  could  query  the  database  for  all  pitfalls  faced  by  an  AC  AT  ID 
program  that  is  heading  to  a  MS  B  decision,  is  expected  to  be  a  joint  service  development, 
aircraft  based,  and  is  in  the  intelligence  functional  domain.  The  ACS  program  information  and 
pitfalls  faced  could  be  used  as  an  initial  set  of  data  for  such  a  database  and  query. 

Continuous  Process  Improvement 

Educating  PMs  on  the  current  potential  pitfalls  in  the  Defense  Acquisition  System  is  only 
a  part  of  the  follow-through  discussed  by  Perkins  above.  The  next  step  is  to  develop  a  method  to 
improve  the  system  by  eliminating  the  errors  in  the  current  acquisition  process  that  cause  pitfalls. 
Based  on  the  specific  knowledge  gained  about  the  issue  areas  of  the  Defense  Acquisition  System 
from  the  PPRs,  focused  improvements  can  be  made  only  to  those  areas  that  need  improvement. 
As  discussed  earlier,  the  entire  acquisition  process  is  not  broken,  but  there  are  errors  in  the 
system  that  must  be  corrected  to  make  the  acquisition  process  more  efficient  and  effective  and 
less  reliant  on  individual  PM  experience.  This  is  not  something  that  can  be  accomplished  by  a 
one-time  set  of  comprehensive  changes  as  has  been  previously  attempted  by  the  numerous 
acquisition  reform  studies.  Instead,  these  improvements  must  be  made  continuously  over  time  to 
be  effective  [82].  The  framework  to  accomplish  such  an  effort  is  found  in  the  Continuous  Process 
Improvement  concept  currently  being  embraced  by  DoD  Business  Transformation[83]. 

On  1 1  May  2006,  the  Deputy  Secretary  of  Defense,  Honorable  Gordon  England  signed  a 
cover  memorandum  to  the  recently  published  Continuous  Process  Improvement  (CPI) 
Transformation  Guidebook.  This  memo  was  addressed  to  all  DoD  subordinate  organizations  and 
mandated  the  establishment  of  Continuous  Process  Improvement  Programs  throughout  DoD.  In 
the  memo  he  stated:  “The  Secretary  and  I  expect  that  every  DoD  organization  is  focused  every 
day  on  improving  the  effectiveness  of  our  support  to  the  Warfighter.”  .  .CPI  has  proven  to  be 
an  important  tool  for  improving  the  operating  effectiveness  of  the  DoD,  not  only  within  the 
logistics  and  acquisition  activities,  but  also  across  the  full  range  of  operational,  administrative, 
Science  and  Technology,  and  support  functions.  We  should  continue  to  broaden  and  accelerate 
the  use  of  these  tools  to  further  improve  effectiveness”[84]. 
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This  memo  and  the  accompanying  CPI  Transformation  Guidebook  are  part  of  an  ongoing 
process  to  change  the  way  DoD  executes  its  mission  that  began  with  the  Secretary  of  Defense’s 
calls  for  DoD  Business  Transformation  back  in  2001  with  release  of  the  Quadrennial  Defense 
Review  (QDR)  of  that  same  year[851. 

.  .And  that  means  we  must  recognize  another  transformation:  the  revolution  in 
management,  technology  and  business  practices.  Successful  modern  businesses  are 
leaner  and  less  hierarchical  than  ever  before.  They  reward  innovation  and  they  share 
information.  They  have  to  be  nimble  in  the  face  of  rapid  change  or  they  die.” 

-  Donald  H.  Rumsfeld,  Secretary  of  Defense,  September  10,  2001  [861 

The  QDR  finds  its  roots  in  the  words  of  President  Bush’s  FY2002  President’s 
Management  Agenda  that  called  for  reform  across  the  Executive  Branch  of  Government  in  a 
effort  to  “improve  federal  management  and  deliver  results  that  matter  to  the  American  people.” 
In  this  agenda  “five  government-wide...  and  nine  agency  specific  reform”  initiatives  are 
discussed  in  results  based  framework.  For  each  initiative:  the  problem  is  presented,  followed  by 
the  initiative  to  fix  the  problem,  then  the  expected  short-term  and  long-term  “results”  or  metrics 
by  which  to  measure  the  initiatives  success. 

This  call  for  transformation  continues  in  the  latest  2006  QDR  where  the  concepts  for 
reform  have  been  refined  with  more  focus  provided  about  how  to  proceed.  Organizations  such  as 
the  Defense  Business  Transformation  Agency  have  been  established  to  manage  and  coordinate 
DoD  transformation  efforts  to  provide  unity  and  leadership  for  the  process  improvement  efforts. 
The  Agency’s  focus  is  .  .on  bringing  the  needed  capabilities  to  the  joint  force  more  rapidly,  by 
fashioning  a  much  more  effective  acquisition  system  and  associated  set  of  processes” [87]. 
Additionally,  the  2006  QDR  calls  for  specific  reforms  to  the  acquisition  process  based  on  the 
DAPA  report.  For  instance,  the  QDR  calls  for  the  reduction  of  cycle-time  to  deliver  a  product 
stating,  “acquisition  development  and  procurement  will  shift  to  a  time-certain  approach.”  It  also 
requires  that  cost,  schedule  and  performance  will  be  aligned  “early  in  program  development, 
senior  leaders  will  make  the  key-tradeoffs  necessary  to  balance  performance,  time  and  available 
resources.”  The  2006  QDR  further  echos  the  specific  reform  initiatives  called  for  by  the  DAPA 
report  by  stating:  “Upgrades  and  improvements  can  be  added  in  subsequent  spirals  based  on  the 
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maturity  of  the  technology.  Combining  time-certain  development  and  procurement  of  capability 
with  a  risk-based  approach  to  source  selection  should  provide  much  greater  stability  in  the 
acquisition  system.  Stability  should  allow  for  more  predictable  acquisition  programs  measured 
by  cost,  schedule  and  performance”  [8  81. 

The  President’s  Performance  Agenda  and  the  QDR  provide  the  impetus  for  reform  and 
demand  that  the  process  of  reform  be  customer  focused  and  results  based.  What  the  use  of  the 
CPI  framework  does  is  to  provide  a  method  within  DoD  to  accomplish  this  customer  and  results 
based  reform.  The  CPI  framework  provides  methods,  when  applied  correctly,  will  force  a 
process  to  be  reshaped  to  provide  value  and  meet  the  warfighter’s  needs.  CPI  also  forces  the 
process  to  become  metric  based  to  allow  success  to  be  measured  and  provide  information  that 
defines  how  process  improvement  should  proceed.  Customer  focus  and  measurements  based 
process  improvement  are  key  components  of  CPI[89]. 

“DoD  CPI  is  a  strategic  approach  for  developing  a  culture  of  continuous  improvement  in 
the  areas  of  reliability,  (reduction  in)  process  cycle  times,  costs  in  terms  of  less  total  resource 
consumption,  quality,  and  productivity”[90].  To  accomplish  this,  CPI  practitioners  have  several 
tools  available  to  use  depending  on  the  nature  of  the  process  to  be  improved.  Lean,  Six  Sigma, 
and  Lean  Six  Sigma  are  several  of  the  tools  available.  To  use  these  tools  effectively  the  proven 
five  phased  model  of  Define,  Measure,  Analyze,  Improve,  Control  (DMAIC)  is  used  to  give  the 
process  improvement  team  a  framework  for  problem  solving  as  they  apply  the  process 
improvement  tools. 

Lean  is  a  tool  that  focuses  on  speeding  up  the  cycle-time  to  delivery  of  a  product.  This  is 
accomplished  by  identifying  and  appropriately  eliminating  “waste  or  non-value  added  activities 
from  a  customers  perspective.  . .  .Waste  is  defined  as  the  activity  or  activities  that  a  customer 
would  not  want  to  pay  for  and/or  that  add  no  value  to  the  product  or  service  from  the  customer's 
perspective.  At  the  heart  of  lean  is  the  determination  of  value.  Value  is  defined  as  an  item  or 
feature  for  which  a  customer  is  willing  to  pay.  All  other  aspects  of  the  process  are  deemed  waste. 
Lean  framework  is  used  as  a  tool  to  focus  resources  and  energies  on  producing  the  value-added 
features  while  identifying  and  eliminating  non  value  added  activities”[911. 

Six  Sigma  is  a  tool  designed  to  eliminate  errors  in  process.  To  accomplish  this,  Six  Sigma 
“measures  how  many  defects  exist  in  a  business  process  and  then  systematically  determines  how 
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to  remove  them.  Its  focus  is  on  process  quality.  . .  .The  principles  of  quality  applied  in 
implementing  Six  Sigma  are  almost  always  defined  in  terms  of  the  company  vision  and  its 
strategy.  Processes  are  designed  from  the  perspective  of  the  customer  and  involve  an  infusion  of 
process  thinking  across  the  firm.  Metrics  such  as  performance,  reliability,  price,  on-time 
delivery,  service  and  accuracy  provide  the  targets”[92]. 

Lean  Six  Sigma  is  a  combination  of  both  tools.  The  thought  is  that  successful  process 
improvement  requires  the  complementary  nature  of  the  two  tools.  Specifically,  Lean  provides  the 
“increase  in  the  velocity”  of  the  process  and  Six  Sigma  provides  the  “quality”  in  the  process. 
“The  fusion  of  Lean  and  Six  Sigma  improvement  methods  is  required  because: 

-  Lean  cannot  bring  a  process  under  statistical  control 

-  Six  Sigma  alone  cannot  dramatically  improve  process  speed  or  reduce  invested  capital 

-  Both  enable  the  reduction  of  the  cost  of  complexity”[93]. 

Solution 

By  combining  PPRs  with  the  CPI  process/tools  and  using  them  to  assess  the  Defense 
Acquisition  System,  DoD  can  solve  its  problem  of  being  unable  to  predict  cost,  schedule,  and 
performance.  As  stated  above,  PPRs  when  executed  correctly,  will  define  the  errors/pitfalls 
within  the  current  Defense  Acquisition  System.  Using  the  PPR  method  of  problem  definition 
provides  an  understanding  of  the  systems  pitfalls  from  the  perspective  of  two  important 
customers  of  the  Acquisition  Process — the  warfighter  and  acquisition  professional.  Hence  using 
PPR  provides  the  first  phase  (Problem  Definition)  of  the  problem-solving  framework  within  CPI 
(DMAIC).  With  the  problem  appropriately  defined,  the  remaining  phases  of  the  CPI  problem¬ 
solving  framework  can  be  implemented.  This  is  accomplished  by  using  CPI  tools  (Lean,  Six 
Sigma,  or  Lean  Six  Sigma)  to  focus  efforts  on  what  needs  to  be  measured,  analyzed,  improved 
and  controlled.  Iterations  of  the  CPI  process  will  ultimately  improve  the  Defense  Acquisition 
System  and  provide  an  Defense  Acquisition  Process  that  can  reliably  predict  cost,  schedule  and 
performance. 
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Immediate  Process  Improvement 

Considering  the  pitfalls  encountered  by  the  ACS  program  during  program  execution — 
discovered  by  reviewing  program  events  both  during  and  post  contract  execution — one  easy 
pitfall  eliminating  process  change  to  the  Defense  Acquisition  System  stands  out.  This  change  is 
to  require  that  all  ACAT  ID  programs  have  an  Independent  Program  Readiness  Assessment 
(IPRA),  conducted  by  experienced  senior  acquisition  personnel,  accomplished  before  they  are 
permitted  to  proceed  to  a  Milestone  decision.  This  change  takes  advantage  of  the  key  lesson 
learned  from  not  only  the  ACS  PPRs  which  demonstrated  the  benefits  of  a  synergistic  review  of 
a  program’s  readiness  to  proceed  to  the  next  acquisition  phase,  but  also  the  many  acquisition 
reform  studies  over  the  years  that  recognize  that  experience  plays  a  major  role  in  a  program 
manager’s  ability  to  understand  his  environment  and  foresee  the  potential  pitfalls  that  could 
impact  his  program.  Additionally,  the  independent  nature  of  the  review  will  ensure  that  program 
politics  and  emotion  do  not  cloud  the  determination  of  a  program’s  readiness  to  proceed  to  the 
next  phase  of  the  acquisition  process.  The  independent  assessment  team  would  execute  an 
experienced  based  review  of  the  program  and  provide  the  PM  and  his  PEO  with  its  synergistic 
assessment  of  the  program’s  risks  that  could  impact  successful  execution  if  realized.  With  this 
knowledge,  a  program  manager  can  take  action  to  avoid  or  mitigate  the  potential  pitfalls 
allowing  him  to  accurately  establish  and  then  maintain  program  cost,  schedule  and  performance. 

Summary  &  Conclusion 

The  Defense  Acquisition  System  remains  intolerably  undependable  in  its  ability  to 
deliver  needed  capability  within  cost,  and  on  schedule  to  its  most  important  customer — the 
Warfighter.  This  situation  is  not  new  and  has  been  the  subject  of  many  Acquisition  Process 
reform  studies  over  the  past  35  years.  The  implementation  of  these  initiatives  has  resulted  in  the 
current  best  practices  based  Defense  Acquisition  System.  Despite  the  improvements  in  the 
Acquisition  Process,  the  most  recent  studies  continue  to  find  the  same  basic  issue  of  a  process 
being  unable  to  deliver  capability  at  estimated  cost  and  schedule.  Because  the  reform  study 
initiatives  have  never  been  fully  implemented,  the  Acquisition  Process  continues  to  be  fraught 
with  potential  pitfalls  that  still  plague  large  MDAP  programs.  The  Congress  noted  in  their  2007 
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National  Defense  Authorization  Act  that  for  many  reasons  they  do  not  expect  this  situation  to 
change.  Despite  the  current  situation,  some  programs  are  successful. 

Reform  studies  and  Post  Project  Reviews,  to  include  the  review  of  the  recent  ACS 
program,  have  shown  that  experience  appears  to  hold  the  key  to  recognizing  potential  pitfalls 
that  cause  program  breaches  and  mitigating  them  before  they  are  realized.  Based  on  this,  DoD 
should  conduct  PPRs  on  all  ACAT  ID  programs  as  they  complete  phases  of  the  acquisition  life- 
cycle  and  Milestone  decisions.  The  identified  pitfalls  of  the  Defense  Acquisition  Process  should 
then  be  placed  in  a  database  and  used  to  train  inexperienced  PMs  on  potential  program  risk  areas. 
This  database  should  also  be  the  foundation  for  the  development  of  a  Program  Expert  System 
that  sitting  PMs  can  access  as  part  of  their  program  planning  process  to  identify  relevant 
potential  pitfalls  that  they  may  not  normally  recognize  based  on  their  past  experiences.  The 
pitfalls  recognized  in  the  ACS  program  should  be  used  to  populate  the  database  with  those 
pitfalls  that  can  occur  in  an  ACAT  ID  program  as  it  transitions  from  TD  to  SDD. 

Calls  for  Business  Transformation  in  the  Executive  Branch  and  within  the  DoD 
specifically,  have  set  the  stage  and  established  the  requirement  to  use  the  framework  for  process 
improvement  known  as  Continuous  Process  Improvement.  CPI  when  combined  with  the  PPR 
process  enables  the  identification  of  pitfalls  within  the  current  best  practices  based  Defense 
Acquisition  System  and  then  provides  a  methodology  to  fix  them.  Specifically,  this  combination 
provides  the  information  and  tools  necessary  to  correct  the  pitfalls  in  the  current  Defense 
Acquisition  System  to  ensure  that  programs  can  meet  cost,  schedule  and  performance. 
Additionally,  DoD  should  establish  an  experienced  team  of  DoD  personnel  who  represent  all 
facets  of  the  “big  A”  Defense  Acquisition  System  and  are  trained  in  the  art  of  PPR  and  CPI  to 
conduct  an  ongoing  analysis  of  the  Defense  Acquisition  System,  identify  its  potential  pitfalls  and 
establish  approaches  and  policy  to  eliminate  these  pitfalls  from  the  Acquisition  Process.  By 
doing  this,  DoD  will  ultimately  be  able  to  regain  the  confidence  of  the  senior  leaders,  Congress 
and  most  importantly  the  Warfighter  since  the  Defense  Acquisition  System  will  finally  have  the 
ability  predict  and  execute  MDAP  programs  on  cost,  within  schedule  and  at  the  required  level  of 
performance. 
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